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Abstract: 


JTL A 


In a pilot study of the Technology Immersion model, high-need middle schools were 
“immersed” in technology by providing a laptop for each student and teacher, wireless 
Internet access, curricular and assessment resources, professional development, and 
technical and pedagogical support. This article examines the fidelity of model implemen- 
tation and associations between implementation indicators and student achievement. 
Results across three years for 21 immersion schools show that the average levels of school 
support for Technology Immersion and teachers’ Classroom Immersion increased slightly, 
while the level of Student Access and Use declined. Implementation quality varied across 
schools and classrooms, with a quarter or fewer of schools and core-content classrooms 
reaching substantial implementation. Using hierarchical linear modeling, we found that 
teacher-level implementation components (Immersion Support, Classroom Immersion) 
were inconsistent and mostly not statistically significant predictors of student achieve- 
ment, whereas students’ use of laptops outside of school for homework and learning 
games was the strongest implementation predictor of achievement. 
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Introduction 

The Technology Immersion Pilot (TIP), created by the Texas Legislature 
in 2003, assumed that the use of technology in Texas public schools could 
be achieved more effectively by “immersing” schools in technology rather 
than by introducing technology resources in a cyclical fashion over time. 
The Texas Education Agency (TEA) has invested more than $20 million 
in federal Title II, Part D monies to fund Technology Immersion projects 
at high-need middle schools through a competitive grant process (i.e., a 
laptop computer for every student and teacher, wireless access throughout 
the campus, curricular and assessment resources, professional develop- 
ment, and ongoing technical and pedagogical support). Concurrently, 
a comprehensive, four-year research study partially funded by a federal 
Evaluating State Educational Technology Programs grant has investigated 
the effects of Technology Immersion on teachers and teaching, students 
and learning, and student achievement. The study’s quasi-experimental 
research design included comparisons between 21 treatment schools and 
21 control schools that enrolled Grades 6 to 8 students. The study also 
has examined the extent to which the Technology Immersion model was 
implemented as designed. The present article focuses on the 21 treatment 
schools’ progress in implementing Technology Immersion and the associa- 
tions between the strength of implementation and students’ reading and 
mathematics test scores. 
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Technology Immersion Model 

State statute (Senate Bill 396, 78 th Texas Legislature) described 
Technology Immersion generally, but to advance consistent interpreta- 
tion of the Technology Immersion model at schools, the TEA issued a 
Request for Qualifications for commercial vendors to apply to become pro- 
viders of Technology Immersion packages. Vendors’ plans had to include 
six components: (a) a wireless mobile computing device for each educator 
and student on an immersed campus; (b) productivity, communication, 
and presentation software; (c) online instructional resources supporting 
the state curriculum in language arts, mathematics, science, and social 
studies; (d) online assessments to diagnose students’ mastery of the core 
curriculum; (e) professional development designed to help teachers inte- 
grate technology into teaching, learning, and the curriculum; and (f) ini- 
tial and ongoing technical support. Through an expert review process, the 
TEA selected three lead vendors to provide Technology Immersion pack- 
ages (Dell Computer Inc., Apple Computer Inc., and Region 1 Education 
Service Center [ESC]). Package costs, which ranged from about $1,100 
to $1,600 per student, varied according to the numbers of students and 
teachers, the type of laptop computer, and the vendor provider. 

Of the 21 Technology Immersion schools studied in the evaluation, 5 
middle schools selected the Apple package, 15 selected the Dell package, 
and 1 school selected the Region 1 ESC package (with Dell computers). 
Table 1 provides an overview of the basic components within each package 
and the individual vendors that provided various products. All vendors 
offered a wireless laptop as the mobile computing device (Apple or Dell), 
and all laptops had a suite of productivity tools (either AppleWorks or 
Microsoft Office). Dell computers also had a web-based portal to applica- 
tions and resources ( eChalk ). Immersion packages included a variety of 
digital curricular resources and formative assessments. Additionally, each 
vendor provided professional development as well as ongoing technical 
support. Apple had its own professional development model. Dell relied 
on a commercial provider ( Pearson Learning Group) and the Dell Exchange 
(an online resource). Region 1 ESC used a combination of service center 
support plus services offered through Connected Coaching and Connected 
University. See Appendix A for a more comprehensive description of the 
package components. 
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Table 1 : Technology Immersion Packages 


Component 

Apple Computer 
N- 5 Schools 
1,625 Students 

Dell Computer 
N= 15 Schools 
5,993 Students 

Region 1 ESC 
N= 1 School 
100 Students 

Wireless laptop computer 

Apple iBook G4 

Dell Inspiron or 
Latitude 

Dell Inspiron 

Productivity software 

Apple Works 

Microsoft Office, 
eChalk 

Microsoft Office, 
eChalk 

Online resources 

Various 

Various 

Various 

Online assessment 

AssessmentMaster 

i-Know 

i-Know 

Professional development 

Apple model 

Pearson Learning 
Group, Dell Exchange 

ESC 1, Classroom 
Connect 

Technical and pedagogical 
support 

Apple, Campus/District 

Dell, Campus/District 

ESC 1, Campus/District 


Note. ESC = Education Service Center. Student enrollment in middle schools (Grades 6, 7, and 8) from Texas 
Education Agency AEIS reports 2005. 


During the third and fourth implementation years, schools began to 
selectively purchase online resources and assessments according to their 
perceived needs. For example, some schools dropped the online assess- 
ments because they had state-provided or local assessments that filled 
their testing needs. Two schools (with Dell and ESC 1 packages) purchased 
the My Access Writing program included in the Apple package. Schools and 
teachers also continued to supplement package resources with products 
purchased locally, provided through state textbook adoptions, or obtained 
from the Internet free of charge. 

The Theoretical Framework for Technology Immersion guided the evalu- 
ation (Figure 1). The framework postulates a linear sequence of causal 
relationships. First, treatment schools are to be “immersed” in technology 
through the implementation of Technology Immersion components. 
An improved school environment for technology is expected to produce 
teachers who are more technically proficient, use technology for profes- 
sional productivity, have students use technology in their classes, and 
use laptops and digital resources to increase the intellectual rigor of les- 
sons. In turn, changed school and classroom conditions are expected to 
improve students’ technology proficiency, learning experiences, collabora- 
tive interactions with peers, personal self-direction, and engagement in 
school and learning. Changes in students and their learning experiences 
presumably contribute to increased academic performance as measured 
by standardized test scores. In the framework, prior student achievement 
and student, family, and school characteristics exert their own influence 
on learning. 
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Figure 1 Theoretical Framework for Technology Immersion 



Prior Research on Implementation 

Beginning in the 1970s, researchers and educators began to recog- 
nize the consequences of failed implementation (Fullan, 1993). Studies of 
large-scale educational reforms and organizational change revealed that 
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the level and quality of implementation largely determined the achieve- 
ment of desired outcomes (Berman & McLaughlin, 1978; Borman, Hewes, 
Overman, & Brown, 2003; Datnow, Borman, 8t Stringfield, 2000; Fullan 
8t Stieglbauer, 1991). Studies of technology innovations, likewise, have 
shown that ineffective implementation undermines prospects for changes 
in student learning opportunities and academic outcomes (Cuban, 2001). 
Given mounting evidence, Desimone stressed the importance of measuring 
“the degree of implementation before assessing outcomes and attempting 
to attribute them to a specific program” (Desimone, 2002, p. 437). 

Recent school change efforts have focused on transforming the whole 
school as a way to improve teaching, learning, and student outcomes. 
Increasing numbers of schools, particularly low performing ones, have 
undertaken comprehensive school reform. Spurred by the availability of 
federal funds and the No Child Left Behind Act of 2001, hundreds of com- 
prehensive school reform (CSR) models have been developed and imple- 
mented in schools. As a result, the nature of model implementation and 
the school change process have been studied widely. Although Technology 
Immersion and CSR models are not exactly the same, they share the 
common focus of changing the whole school, including changes to the cur- 
riculum and delivery of instruction. Thus, past research contributes to a 
clearer understanding of implementation for this study. 

In-depth studies of CSR model implementation demonstrate that 
achieving quality implementation is challenging. Studies involving hun- 
dreds of CSR schools revealed that none of the schools had fully imple- 
mented all components of the models they had adopted. Schools appeared 
to implement components selectively (Kurki, Aladjem, 8t Carter, 2005). 
And notably, schools had more difficulty following instructional prac- 
tices prescribed by their model and practices aimed at increasing parental 
involvement in school affairs (Vernez, Karam, Mariano, & DeMartini, 
2006). In these studies, higher levels of implementation were associated 
with higher levels of support (e.g., principal leadership, teachers’ com- 
mitment, model developer support, professional development). Findings 
regarding the influence of contextual variables (school size or student 
characteristics) were mixed as were results regarding the improvement of 
implementation over time. Kurki et al. (2005) cited increased implemen- 
tation for most indicators between the first and third year, whereas Vernez 
et al. (2006) found that the first year or so, for the most part, determined 
the degree of implementation, with levels remaining fairly constant across 
three years. 

Evidence from a meta-analysis of CSR student achievement effects 
shows that the quality of implementation matters. Using the best avail- 
able measure of implementation — the number of years a CSR model 
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was implemented in a school— researchers found an increasing effect on 
achievement outcomes associated with a greater number of years of imple- 
mentation. The CSR effect size was relatively strong in the first year (0.17), 
but there was a tendency for new initiatives to weaken in the second, third, 
and fourth years. On the other hand, schools that implemented models for 
five or more years showed achievement advantages nearly twice as large 
(Borman et al., 2003; Borman, 2005). These long-term effects, however, 
may be positively biased by the large proportions of schools (up to one 
third) that discontinued use of CSR models within the first few years, and 
consequently, were excluded from longitudinal analyses (Borman, 2005). 

Researchers who have studied one-to-one laptop initiatives have also 
examined the quality of implementation and factors that affect program 
success. Although the goals and scale of one-to-one laptop projects vary 
widely, there is growing consensus that effective implementation requires 
a comprehensive or systemic approach that includes attention to aspects 
such as leadership and planning, supportive school culture, training and 
professional development, robust infrastructures and technical support, 
and access to digital content and instructional resources (Zucker, 2005). 
Specifically, committed leadership (at the state, district, school, and class- 
room levels) has been associated with stronger implementation. Effective 
leaders articulate a compelling vision of how laptops advance teaching 
and learning, develop policies and procedures that support change, foster 
collaborative environments, and marshal needed resources (Bradburn & 
Osborne, 2007; Pitler, 2005; Zucker, 2005; Zucker & McGhee, 2005). 

Teachers’ attitudes and beliefs about technology can also affect imple- 
mentation (Penuel, 2006). Several studies report that the use of technology 
in a one-to-one classroom reflects teachers’ beliefs about their students 
and the potential of technology for learning, as well as their conception of 
what constitutes effective teaching (Garthwait & Weller, 2005; McGrail, 
2006; Russell, Bebell, Cowan, & Corbelli, 2002; Windschitl & Sahl, 2002). 
One study cited factors that diminished teachers’ commitment to one-to- 
one computing, including top-down institutional decision making about 
the program, conflicts between technology use and test-preparation 
pressures, and conflicts about the fit of technology with the curriculum 
(McGrail, 2006). 

Given teachers’ important role in implementation, high-quality sus- 
tained professional development is a critically important factor. Studies 
cite the need for teacher professional development that builds teachers’ 
basic technology skills as well as their understanding of curricular integra- 
tion (Lowther, Ross, & Morrison, 2001; Ringstaff & Kelley, 2002; Owen, 
Farsaii, Knezek, & Christensen, 2005-06). Teachers also need follow-up 
support as they acquire and implement new skills in the instructional 
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setting (Bradburn & Osborne, 2007; Neugent & Fox, 2007; Owen et al., 
2005-06). Informal assistance from colleagues may also advance imple- 
mentation, with teachers helping each other solve technical problems and 
sharing ideas about lessons (Silvernail & Harris, 2003; Windschitl & Sahl, 
2002). Projects that reportedly have influenced teachers’ beliefs about the 
value of laptops provided professional development that gave teachers 
a framework to develop problem-based lessons (Lowther et al., 2003), 
required teachers to engage in projects with students (Light, McDermott, 
& Honey, 2002), and provided resources that supported teachers’ par- 
ticular content area (Lane, 2003). Readily available technical support and 
dependable wireless networks also have been linked with implementation 
success (Lane, 2003; Silvernail & Harris, 2003; Silvernail & Lane, 2004; 
Zucker & McGhee, 2005). As a whole, findings point to the crucial need 
for researchers to measure the extent of implementation in evaluating the 
effectiveness of one-to-one laptop interventions. 

Purpose of the Current Study 

The research presented in this article represents one part of the overall 
evaluation ofTechnology Immersion. For the current study, we focused on 
the implementation aspect of the theoretical framework (i.e., the fidelity 
with which the components of Technology Immersion attained the model’s 
envisioned ideal). In particular, we investigated the extent to which each of 
the 21 treatment schools implemented the Technology Immersion model as 
designed and assessed each school’s progress across three implementation 
years. Additionally, given variations in implementation, we investigated 
the relationship between implementation strength (at the school, teacher, 
and student levels) and students’ reading and mathematics achievement 
as measured by scores on the state’s criterion-referenced assessment — the 
Texas Assessment of Knowledge and Skills (TAKS). 
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Method 

Participants 

Schools 

In spring 2004, interested districts and associated middle schools 
responded to a Request for Application (RFA) offered by the TEA to become 
Technology Immersion schools. Applicants had to meet eligibility require- 
ments for Title II, Part D funds (i.e., high need due to children from families 
with incomes below the poverty line, schools identified for improvement, 
or schools with substantial need for technology). The agency held an 
external review of proposals, with applications scored and rank ordered. 
Following the external review, researchers and agency staff reviewed pro- 
posals to ensure that applications met criteria established for Technology 
Immersion. Final selection of schools involved the consideration of sev- 
eral factors, including proposal ratings, size, location, student diversity, 
and academic achievement. Decisions were influenced by the need for 
geographic distribution and the availability of comparable schools for the 
control group pool. Schools received grants to support the implementa- 
tion of Technology immersion for four school years (2004-05, 2005-06, 
2006-07, and 2007-08). Twenty-one middle schools with Grades 6 to 8 
students (treatment group) were drawn from rural, suburban, and urban 
locations across Texas. Middle schools were typically small, with more than 
three-quarters enrolling 600 students or fewer. The majority of schools 
were concentrated in small or very small school districts (2,999 or fewer 
students), but a third of schools were in large districts (10,000 or more 
students). 
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Teachers and Students 

As Table 2 shows, nearly 600 teachers worked at Technology Immersion 
schools each year. Across four years, middle-school teachers were primarily 
female (about 66%), White (about 55%) or Hispanic (about 40%), and they 
were fairly experienced (about 11 years teaching, on average). About a fifth 
of teachers had advanced degrees. 


Table 2: Demographic Characteristics of Teachers at 

Technology Immersion Schools by Year 



2004-05 
Year 1 

2005-06 
Year 2 

2006-07 
Year 3 

2007-08 
Year 4 

Number of teachers 

593.0 

604.0 

591.0 

612.0 

% Female 

65.6 

63.4 

66.5 

66.3 

% Minority 

41.2 

44.9 

43.1 

45.9 

% African American 

3.4 

2.8 

3.2 

4.4 

% Hispanic 

35.3 

40.4 

39.9 

39.3 

% White 

58.7 

55.1 

55.2 

54.1 

% with no degree 

0.9 

0.2 

0.2 

0.3 

% with advanced degree 

21.3 

21.2 

19.3 

19.2 

Average years experience 

10.6 

10.6 

10.8 

11.5 


Note. Year 1 was the start-up year. Researchers developed implementation fidelity measures during that year. 
Statistics from Texas Education Agency AEIS reports. 
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Statistics in Table 3 show that most of the Technology Immersion stu- 
dents were economically disadvantaged (more than three-quarters quali- 
fying for federal free or reduced-price lunches), and they were ethnically 
diverse (about 75% Hispanic, 19% White, and 5% African American). More 
than a fifth of students were limited English proficient (LEP). This study 
concentrates on three groups or cohorts of students. Cohort 1 students 
(8th graders in 2006-07) attended Technology Immersion schools across 
the first three project years (Grades 6, 7, and 8) and then enrolled at local 
high schools in the fourth year. Cohort 2 students (8th graders in 2007- 
OS) attended immersion schools for three years (Grades 6, 7, and 8), and 
Cohort 3 students (7th graders in 2007-08) attended immersion schools 
for two years (Grades 6 and 7). A fourth group of sixth graders are included 
in implementation measures for Year 4 (2007-08). 

Table 3: Demographic Characteristics of 

Technology Immersion Students by Cohort 



Cohort 1 
8 th Graders 
2006-07 

Cohort 2 
8 th Graders 
2007-08 

Cohort 3 
7 th Graders 
2007-08 

Number of students 

2,586.0 

2,578.0 

2,547.0 

% Economically disadvantaged 

75.8 

75.5 

76.7 

% African American 

5.8 

5.1 

4.3 

% Hispanic 

72.7 

75.1 

75.9 

% White 

20.4 

18.8 

19.2 

% Lmited English Proficient 

22.7 

20.8 

26.3 

% Female 

48.6 

49.7 

48.4 

% Male 

51.4 

50.3 

51.6 


Note. Statistics from school-provided files for the 2006-07 and 2007-08 school years. 


Data Collection 

Data collection for the study began in August 2004 and continued 
through spring 2008. This analysis focuses on the second (2005-06), 
third (2006-07), and fourth (2007-08) implementation years. Measures 
included teacher and student surveys completed at the end of each school 
year (April to May), and students’ TAKS scores from annual administra- 
tions in April. The technology survey response rates for teachers and stu- 
dents are summarized in Table 4. Survey response rates for teachers ranged 
from 87% to 94% across years. Student survey response rates ranged from 
82% to 89% across years for students who were members of Cohorts 1, 2, 
and 3, and from 82% to 87% for all students completing surveys. 
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Table 4: Teacher and Student Technology Survey Respondents and 

Response Rates by Year 





Students 


Teachers 

Cohort 1 

Cohort 2 

Cohort 3 

All 

Spring 

N 

% 

N 

% 

N 

% 

N 

% 

N 

% 

2006 

560 

92 

2,291 

87 

2,379 

89 



7,022 

87 

2007 

619 

94 

2,168 

84 

2,228 

84 

2,220 

85 

7,832 

85 

2008 

534 

87 



2,110 

82 

2,130 

84 

7,680 

85 


Note. Teacher respondents in 2006, 2007, and 2008 included 318, 371, and 337 core-subject teachers, 
respectively (English language arts/reading, mathematics, social studies, and science teachers). 


Table 5 displays five survey response-rate categories, and the number 
of schools that had teacher or student survey response rates that fell within 
the percentage ranges. Across years, nearly all of the 21 schools had survey 
response rates between 80% and 100%. In 2006 and 2007, a few schools 
had response rates between 60% and 79%. In 2008, the number of schools 
with survey response rates in the 50% to 59% and 60% to 69% ranges 
increased. As might be expected, lower implementing schools tended to 
have lower survey response rates. 

Table 5: Number of Schools within Response-Rate Categories by Teachers, 

Students, and Survey Year 



Teachers 

Students 

Survey response rate 

2006 

2007 

2008 

2006 

2007 a 

2008 

90-100% 

12 

18 

12 

10 

12 

13 

80-89% 

6 

3 

3 

9 

4 

5 

70-79% 

2 

0 

2 

2 

3 

0 

60-69% 

1 

0 

2 

0 

0 

2 

50-59% 

0 

0 

2 

0 

0 

1 


Note. N = 21 Technology Immersion schools. 

a In spring 2007, two schools did not return student surveys. Researchers used model-based imputation 
(AMOS 7.0) to predict a student's 2007 scale score from the spring 2006 score, gender, ethnicity, and 
economic status. 


Teacher Questionnaire 

Teachers completed an online technology survey that included items 
related to school technology, technology use, and professional develop- 
ment activities. Across three years, 560, 619, and 534 teachers, respec- 
tively, completing surveys. For school technology, teachers rated their 
strength of agreement with statements on a 5-point scale ranging from 
1 (strongly disagree) to 5 (strongly agree) for four factors: Leadership (12 
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items), Technical Support (5 items), Innovative Culture (4 items), and 
Parent and Community Support (2 items). Cronbach’s alpha measures of 
internal consistency for school-level factors ranged from 0.66 to 0.97 (see 
Appendix B). Teachers also completed items describing their professional 
development experiences (Contact Hours, Classroom Support, Content 
Focus, and Coherence), which were aggregated at the school level. Alpha 
scale reliabilities ranged from 0.67 to 0.92. 

Surveys also included measures of individual teacher variables. Items 
pertained to teachers’ perceptions of Professional Productivity (11 items), 
Communication (4 items), Student Classroom Activities (17 items), 
Technology Integration (10 items), and Learner-Centered Instruction 
(4 items). Cronbach’s alpha reliability coefficients for scales ranged from 
0.74 to 0.98. For Professional Productivity, Communication, and Student 
Classroom Activities, teachers used a 5-point scale to rate the frequency 
of activities or interactions: 0 (never), 1 (rarely-e.g., a few times a year), 2 
(sometimes-e.g. , once or twice a month), 3 (often-e.g., once or twice aweek), and 
4 (almost daily). Measures of teachers’ ideology — Technology Integration 
and Learner-Centered Instruction — involved a 7-point scale ranging from 
0 (not true of me now) to 6 (very true of me now). 

Student Technology Survey 

Grades 6 through 8 students at treatment schools completed paper- 
and-pencil surveys in spring 2006 (N =7,022), spring 2007 (N = 6,634), 
and spring 2008 (N = 6,327). These responses, which were used to generate 
school-level measures of implementation fidelity, included both cohort 
and non-cohort group students. The individual responses of students in 
Cohorts 1,2, and 3 were used in analyses of the association between imple- 
mentation and academic achievement. Student survey items measured (a) 
the extent of students’ laptop access (items used to calculate the number 
of days out of the 180 day school year that a laptop was available), (b) the 
frequency of laptop use for Core-Content Learning (a 5-point scale used 
to rate the frequency of activities or interactions in English language arts/ 
reading, math, science, and social studies classes: 0 (never), 1 (rarely-e.g., 
a few times a year), 2 (sometimes-e.g., once or twice a month), 3 (often-e.g., 
once or twice a week), and 4 (almost daily), and (c) the extent of laptop use 
at home (0 = no home laptop access, 1 = home laptop access, and up to 5 
additional points for laptop use for homework in language arts (reading/ 
writing), social studies, science, math, and/or for playing games to learn). 
The Cronbach’s alpha reliability coefficient for the Core-Content Learning 
scale was 0.73. 
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Academic Achievement 

The academic outcome measures were TAKS reading and mathematics 
scores. The TAKS is a criterion-referenced assessment that annually 
measures students’ mastery of the state’s content standards. Reading is 
assessed at Grades 3 to 9 and mathematics at Grades 3 to 11. The TAKS 
was first administered in spring 2003. The TEA developed the objectives 
for the assessment program with input and feedback from educators and 
the general public. Test items were developed by test contractors Harcourt 
Assessment and Pearson. A technical advisory committee composed of 
nationally recognized educational testing experts was assembled to pro- 
vide advice for setting performance standards. Internal consistency reli- 
abilities for assessments are in the high 0.80s to low 0.90s range. Evidence 
also supports the content, construct, and criterion-related validity of 
TAKS assessments. Additional information about the technical quality of 
TAKS tests is available in Technical Digests on the TEA website at http:// 
www.tea.state.tx.us/index3 .aspx?id=43268unenu_id3=793. 

The TAKS scale score has a passing standard set at 2100 for each grade 
level. Because scores are not equated across grades, we used TAKS scale 
scores to calculate standardized scores that could be used to measure stu- 
dent progress across grade levels. The standard score is a T score with a 
mean of 50 (state average) and a standard deviation of 10. The achieve- 
ment analyses reported for this study used students’ individual TAKS T 
scores for reading and mathematics as the dependent variables, and each 
student’s TAKS reading or mathematics T score for the previous school 
year was used to account for prior academic achievement. Analyses were 
conducted separately for students in Cohorts 1, 2, and 3 who were con- 
tinuously enrolled in treatment schools across implementation years. 

Measuring Implementation Fidelity 

Implementation was measured as the fidelity with which Technology 
Immersion components and related elements attained the envisioned “ideal.” 
This approach involved gathering data on immersion components at each 
of the treatment schools and comparing school-to-school variations with 
the vision for “full” implementation. The seven immersion components 
(see Table 6 and Appendix B) included five supports for implementation 
(Leadership, Teacher Support, Parent and Community Support, Technical 
Support, and Professional Development) and two components related to 
teacher and student implementation outcomes (Classroom Immersion 
and Student Access and Use). We used a two-part measurement approach. 
First, we used indicators to describe each school’s progress on a 4-stage 
scale toward immersion standards. Rating scales for components and 
related elements identified four levels of immersion: minimal (0 to 1.99), 
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partial (2.00 to 2.99), substantial (3.00 to 3.49), and full (3.50 to 4.00). 
The scoring criteria for the four cut-points were articulated in scoring 
rubrics developed by researchers who had observed treatment schools’ 
implementation progress during site visits that involved interviews, focus 
groups, building walkthroughs, and classroom observations. Researchers 
conducted site visits at baseline (fall 2004) and at the end of each imple- 
mentation year (2005 through 2008). Scoring rubrics guided the assign- 
ment of second-year implementation ratings for each of the 21 treatment 
schools. Next, the scoring rubrics and second-year implementation rat- 
ings for schools were reviewed collectively by researchers, TEA staff and 
agency liaisons who were assigned to individual schools, and vendor repre- 
sentatives who had first-hand knowledge of their schools’ progress. These 
expert judges generally agreed that schools had been accurately placed in 
low-to-high implementation categories. Their comments and suggestions 
for scoring were incorporated into revised versions of the scoring rubrics 
(see Appendix C). 

As a second implementation measure, we used quantitative imple- 
mentation indices to gauge the level of Technology Immersion using stan- 
dardized scores ( z scores). Z scores allowed the calculation of composite 
scores across indicators with varying scales and standard deviations. Table 
6 provides descriptions of the Technology Immersion indicators. The data 
sources used for calculating implementation indicators are detailed in 
Appendix B. Explicitly defined procedures ensured consistent measure- 
ment of implementation indicators from year to year. 
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Table 6: Description of Implementation Indicators for Technology 

Immersion 


Support for Technology Immersion 

Leadership 

To what extent do teachers indicate that administrators establish a clear vision and 
expectations, encourage integration, provide supports, and involve staff in making 
decisions about instructional technology. 

Teacher 

Support 

To what extent do teachers share an understanding about technology use, do 
teachers continually learn and seek new ideas, are teachers unafraid to learn about 
and use technologies, and are teachers supportive of integration efforts. 

Parent and 

Community 

Support 

To what extent do teachers believe that parents and the surrounding community 
support the school's efforts with technology. 

Technical 

Support 

To what extent do teachers indicate that technical problems with computers, Internet 
access, repairs, and material availability pose barriers to Technology Immersion. 

Professional 

Development 

Contact Hours To what extent does the duration (hours) of technoloav-related 
professional development (PD) support the integration of technology into teaching, 
learning, and the curriculum. 

Classroom SuDDort To what extent do core-subiect teachers receive coachina 
or mentoring from an internal source, such as another teacher or technology 
coordinator, or an external (non-school) source. 

Content Focus To what extent do core-subiect teachers indicate that PD emphasizes 
curriculum, instructional methods, and lesson development in core subjects. 

Coherence To what extent do core-subiect teachers indicate that PD is consistent 
with personal and school goals, builds on prior learning, and supports state 
standards and assessments. 

Classroom Immersion 

Technoloav Intearation To what extent do core teachers alter instructional Dractices. allocate 
time, integrate research on teaching and learning, improve basic skills, and support higher order 
thinking through technology. 

Learner-Centered Instruction To what extent do teachers have students establish learnina aoals, 
use information and inquiry skills, complete alternative assessments, and have active and relevant 
learning experiences. 

Student Classroom Activities To what extent do teachers have students use particular technoloav 
resources for learning in core-subject classes, such as a word processor for writing, a spreadsheet 
for calculation or graphing, or the Internet for research. 

Communication To what extent do teachers use technoloav to communicate with students, parents, 
and colleagues or to post information on a class website. 

Professional Productivitv To what extent do teachers use technoloav to enhance their professional 
productivity (e.g., keep records, analyze data, develop lessons, deliver information). 

Student Access and Use 

Laptop Access To what extent do students have access to wireless laptops throuahout the school vear. 

Core-Subiect Learnina Howfreauentlv do students use technoloav resources for learnina in 
core-subject classes. 

Home Learnina To what extent do students have access to and use laptops outside of the school 
for homework and learning. 


Note. See Appendix B for a technical description of the measurement of implementation indicators. 
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Scores for Immersion Standards 

We used teacher and student survey data to compute implementation 
scores for indicators that measured progress toward immersion standards 
(i.e., minimal to full implementation). Adapting a process developed by 
the RAND Corporation, 1 the value for each indicator was computed rela- 
tive to the maximum value (4.00 — the value assigned to full implementa- 
tion). Standardization based on the maximum value allowed comparisons 
across different types of indicators. For each component and element of 
Technology Immersion, standardization involved the following computa- 
tions: 

• Agreement scales (i.e., strongly agree or strongly disagree with a 
prescribed practice or behavior): 4 = strongly agree, 3 = agree, 

2 = neither agree nor disagree, 1 = disagree, and 0 = strongly 
disagree. 

• Frequency scales (i.e., four- or five-level frequencies of doing 
a prescribed practice): 4 = highest frequency met, 3 or 2.67 = 
second highest frequency, 2 or 1.33 = third-highest frequency, 

1 = fourth -highest frequency, and 0 = never or do not do. 

• Continuous variables (i.e., how much time or how often a 
prescribed practice is done): 4 = meet or exceed requirements, 
and 0-3.99 = proportional fraction of requirement. 

We computed the following immersion standard scores on the 0 to 4 
scale. 

• A mean immersion standard score for each Technology 
Immersion support component at each school (e.g., Leadership 
was an average score for 12 items based on the responses of 
all teachers in a school who completed the survey; Professional 
Development was a mean score for core-subject teachers at 

a school based on their average scores for Contact Hours, 

Classroom Support, Content Focus, and Coherence); 

• a mean Classroom Immersion score for each school’s 
core-subject teachers (Classroom Immersion was an average 
score for Technology Integration, Learner-Centered 
Instruction, Student Classroom Activities, Communication, 
and Professional Productivity); and 

• a mean Student Access and Use score for Grades 6, 7, and 
8 students (Student Access and Use was an average score 
for Laptop Access Days, Core-Content Learning, and Home 
Learning). 
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Scores for Implementation Indices 

In addition to the standards-based scoring system described above, we 
used teacher and student survey data to compute standardized implemen- 
tation indicators (z scores) that could be aggregated to generate: 

• a single implementation score for each Technology Immersion 

component for each school (e.g., Leadership Index was an 
average z score for 12 items based on the responses of all 
teachers in a school who completed the survey). 

• a mean Immersion Support z score for the five support 

components (Immersion Support Index was an average 
of Leadership Index, Teacher Support Index, Parent and 
Community Support Index, Technical Support Index, and 
Professional Development Index); 

• a mean Classroom Immersion z score for core-subject teachers in 

a school (Classroom Immersion was an average of Technology 
Integration, Learner-Centered Instruction, Student Classroom 
Activities, Communication, and Professional Productivity); 

• a mean Student Access and Use z score (Student Access and Use 

was an average of Laptop Access, Core-Content Learning, and 
Home Learning); and 

• an overall mean implementation z score for each school 

(Implementation Index was an average of Support Index, 

Classroom Immersion Index, and Student Access and Use Index). 

Data Analyses 

This study combined three analytic approaches to examine the imple- 
mentation ofTechnology Immersion at the 21 treatment schools. First, we 
used descriptive statistics (mean standard scores, percentages of schools at 
implementation levels) to describe how the Technology Immersion model 
and its component parts were implemented. Second, we used implementa- 
tion indices (mean z scores) to create school profiles and to examine asso- 
ciations between implementation components. Finally, we used two-level 
hierarchical linear models (with individual students nested within their 
reading and mathematics teachers) to investigate associations between 
treatment fidelity indicators (z scores) and student academic outcomes. 
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Results 

In sections to follow, we first present results for implementation 
standards (measured at four levels) that describe the extent to which the 
Technology Immersion model’s support components and instructional and 
learning components were implemented as designed. These scores showed 
whether middle schools attained the standards that represented what a 
substantially or fully immersed campus should achieve. Next, findings for 
implementation indices (z scores) provide an overall measure of the level of 
Technology Immersion (Implementation Index) and compare the relative 
level of implementation for individual components across schools. Finally, 
we present analyses of associations between implementation fidelity and 
student academic achievement. 

Implementation ofTechnology Immersion — 
Implementation Standard Scores 

As explained previously, progress toward Technology Immersion 
standards was measured at four levels ( minimal , 0-1.99; partial, 2.00- 
2.99; substantial, 3.00-3.49; and full immersion, 3.50-4.00). Five com- 
ponents assessed the strength of supports for Technology Immersion 
(Leadership, Teacher Support, Parent/Community Support, Technical 
Support, Professional Development), whereas one component gauged the 
extent of teachers’ Classroom Immersion and another component mea- 
sured Student Access and Use (of technology). Figure 2 displays the mean 
implementation scores by component and project year. Mean standard 
scores for components generally showed small increases across years, with 
the exception of Technical Support (which remained fairly stable: 2.73, 
2.82, and 2.76) and Student Access and Use (which declined each year: 
2.17, 2.15, and 2.07). Fourth-year mean implementation support scores 
ranging from 2.69 (Professional Development) to 3.19 (Teacher Support) 
showed that supports for immersion from school administrators, teachers, 
the community, technical staff, and professional development providers 
did not reach full implementation standards (mean score of 3.50 to 4.00). 
Consistent with the second and third project years, teachers, on average, 
reported only partial levels of Classroom Immersion in the fourth year 
(M = 2.67), and students, as a whole, reported partial levels of technology 
access and use (M = 2.07). 
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Figure 2: Mean level of implementation (measured on a 0 to 4 scale) 

for seven Technology Immersion components (N = 21 middle 
schools) by year. 2 



Year 2 
Year 3 
| Year 4 


Results for Year 4, reported in Table 7 and discussed below, showed 
that the level of implementation for individual components varied across 
schools. 
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Table 7: Level ofTechnology Immersion (Year 4) Measured by 

Implementation Standard Scores 



Average 

Implementation 

Percentage of Schools at each 
Implementation Level 

Component 

Mean 

SD 

Minimal 

Partial 

Substantial 

Full 

Immersion Support 

Leadership 

2.98 

0.26 

0 

48 

52 

0 

Teacher Support 

3.19 

0.27 

0 

24 

67 

10 

Parent/Community Support 

2.82 

0.36 

0 

71 

24 

5 

Technical Support 

2.76 

0.36 

0 

67 

29 

5 

Professional Development 

2.69 

0.49 

9 

52 

33 

5 

Classroom Immersion 

2.67 

0.32 

5 

76 

19 

0 

Student Access and Use 

2.07 

0.43 

43 

57 

0 

0 


Note. SD = Standard Deviation. Level of implementation for components measured on a 0 to 4 scale. 


Level of School Support 

Leadership 

Given the importance of administrators’ support for Technology 
Immersion, teachers have been asked to rate the quality of administra- 
tive leadership at their schools. Administrators demonstrated leadership 
through behaviors such as involving staff in decisions, setting clear expec- 
tations for technology use, encouraging and participating in professional 
development events, and providing resources and support. Results in 
Figure 2 showed that administrative leadership was relatively stable across 
three years. Percentages reported in Table 7 for the fourth year show that 
teachers at about half of schools (52%) reported substantial levels of lead- 
ership, which indicated that these teachers either agreed or strongly agreed 
that administrators provided technology-related leadership. Teachers in 
an additional half of schools (48%) reported just partial levels of adminis- 
trative support. 

Teacher Support 

Teacher “buy-in” for Technology Immersion is critically important 
because students’ school experiences with technology are largely dic- 
tated by their teachers. Thus, it was noteworthy that teachers reported 
increased levels of support for technology innovation across years. In the 
fourth year, teachers at two campuses (10%) reported a full level of sup- 
port. That is, teachers at these schools strongly agreed that they shared an 
understanding about technology use for student learning, were continu- 
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ally learning and seeking new ideas, were not afraid to learn about and 
use new technologies, and were supportive of integration efforts. Teachers 
at two-thirds of schools (67%) reported a substantial level of support for 
technology innovation, whereas teachers at a quarter of campuses (24%) 
reported only partial levels of support. 

Parent and Community Support 

Because parents must share responsibility for an expensive laptop 
computer with their child or children, their understanding of and support 
for Technology Immersion is imperative. Additionally, the enthusiastic 
support of community members, including elected members of the local 
school board and business people, may influence implementation through 
mechanisms such as the adoption of supportive policies or provision of 
financial resources. In the fourth year, teachers at about a third of schools 
reported substantial levels (24%) to full levels (5%) of parent and commu- 
nity support, with teachers generally agreeing that parents and the sur- 
rounding community supported their efforts with technology. Conversely, 
teachers at 71% of schools reported just partial levels of parent and com- 
munity support. Thus, garnering adequate parent and community support 
remained a problem at many schools in the fourth year. 

Technical Support 

Technical support for immersion should be provided by vendor tech- 
nicians as well as district and campus staff who assist with implemen- 
tation and offer timely support when technical problems arise. Results 
showed that the average level of technical support was similar across 
years. However, results for the fourth year showed that teachers at about 
a third of schools reported substantial levels (29%) or full levels (5%) of 
technical support, whereas teachers at two-thirds of schools reported just 
partial levels of technical support (67%). Teachers at schools with par- 
tial implementation were generally unsure that school computers were 
kept in working order, requests for assistance were addressed in a timely 
way, Internet connections worked adequately, and classroom materials 
were readily available. Clearly, technical problems continued to challenge 
teachers at many schools in the fourth year. 

Professional Development 

Technology Immersion packages included a professional development 
component designed to support all educators on an implementing campus. 
The immersion model required professional development that instructed 
teachers in effective classroom integration and was delivered through 
proven methods (i.e., learning through a variety of delivery systems, col- 
laboration, sustained learning opportunities, and ongoing coaching and 
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support). In addition to professional development provided by immer- 
sion package vendors, each school was offered a grant in the fourth year 
to participate in the Intel Teach Program (i.e., grant funds to train Intel 
Master Teachers [MTs] and stipends for at least 10 participant teachers to 
be trained by the MTs). Of the 21 treatment schools, 17 schools received 
grants to train at least one MT who provided school-based training for 
their peers. 

Although professional development was for all teachers at a school, our 
implementation measure concentrated on core-subject teachers because of 
their close association with measured student academic outcomes. Year- 
to-year comparisons displayed in Figure 2 for the composite Professional 
Development indicator (mean score for four standards-based elements) 
showed that the level of professional development improved slightly in the 
fourth year (likely due to the Intel program). Even so, fourth-year results 
displayed in Table 7 showed that the majority of campuses had minimal 
(9%) to partial (52%) levels of professional development, while a third of 
campuses achieved substantial (33%) or full (5%) levels of professional 
development. 

Figure 3 compares the implementation levels for each of the four ele- 
ments that contributed to the composite Professional Development mea- 
sures. Despite annual increases in Contact Hours, core teachers reported 
receiving less than the prescribed number of hours of technology-related 
professional development (50 or more hours per year). The mean imple- 
mentation score for Year 4 (2.79) indicated that teachers, on average, 
participated in 37 hours or fewer of technology-related professional devel- 
opment. Additionally, teachers, reported just partial levels of classroom 
support indicating that teachers as a whole rarely (a few times a year) or 
never received classroom coaching or mentoring from an internal source 
(such as another teacher or technology coordinator) or external source 
(such as a vendor-provided professional trainer). Moreover, teachers’ mean 
rating for Coherence indicated that professional development was consis- 
tent with their personal goals and experiences to a minimal extent (partial 
implementation). Core-subject teachers expressed stronger agreement 
about the content-focus of professional development. The mean fourth- 
year score (3.09) indicated that teachers’ professional development had a 
minor to major emphasis on curriculum, instructional methods, and lesson 
development in core-content areas. 
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Figure 3: Level of Implementation (measured on a 0 to 4 scale) 

for elements of the Professional Development component 
by mean implementation score and year 

“1 | Year 2 

■ Year 3 



Level of Classroom Immersion 

Given the needed equipment, digital resources, and support for 
Technology Immersion, teachers are expected to design technology- 
enhanced learning environments and integrate technology into teaching, 
learning, and the curriculum. Fourth-year results for teachers’ com- 
posite level of Classroom Immersion (Table 7) showed that core-subject 
teachers at about a fifth of schools (19%) had reached substantial levels 
of Classroom Immersion. On the other hand, teachers at the majority of 
schools reported only partial levels of Classroom Immersion (76%), and 
teachers at one school (5%) had a minimal level of Classroom Immersion 
in the fourth year. 

Figure 4 illustrates teachers’ implementation progress relative to the 
five elements of Classroom Immersion: Technology Integration, Learner- 
Centered Instruction, Student Classroom Activities (with technology), 
Communication, and Professional Productivity. On average, teachers 
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reported partial levels of implementation across years for four of five ele- 
ments. Teachers’ technology use for their own Professional Productivity 
reached a substantial level of implementation in the fourth year (M = 3.04). 
Comparisons across years indicated that teachers, on average, became 
somewhat more positive about technology integration, learner-centered 
instructional methods, and the use of technology as a communication 
and professional productivity tool. In contrast, the frequency with which 
core-subject teachers had students in their classrooms use technology for 
learning activities remained fairly stable across years. In general, teachers 
at many schools seemed to view technology as a more valuable tool for 
themselves than for their students. 

Figure 4: Level of Implementation (measured on a 0 to 4 scale) for five 

elements of Classroom Immersion by mean implementation score 
and year 

| Year 2 
Year 3 
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Level of Student Technology Access and Use 

The transformation of students’ classroom experiences is a vital part 
of Technology Immersion; however, the model also aims for students to 
have access to technology outside of school that allows them to become 
more independent and self-determined learners. Overall, data reported 
by students indicated that Student Access and Use remained relatively 
stable across the second and third project years but declined slightly in 
the fourth year. Between the second and fourth years, the percentages of 
schools with partial levels of Student Access and Use decreased (from 76% 
to 57%, respectively) whereas the percentages of schools with minimal 
access and use increased (from 24% to 43%, respectively). None of the 
schools reached substantial to full levels of Student Access and Use. 

Figure 5 shows the average implementation levels for three elements: 
Laptop Access Days, Core-Content Learning, and Home Learning. First, in 
a fully immersed school, all students should have access to their wireless 
laptops and resources nearly the entire school year (about 170 to 180 days). 
Schools as a whole, however, had difficulty keeping laptops in the hands of 
students. Year-to-year comparisons indicated that students’ Laptop Access 
Days declined in the third year (from 2.69 to 2.50) but improved in the 
fourth year (2.64). Thus, students, on average, had laptops available for a 
larger number of days in the fourth year. Students’ access to laptops was 
affected by factors such as time for repairs, technical issues, disciplinary 
infractions, and parent resistance. In contrast to laptop access, students’ 
use of their laptops in English/language arts, mathematics, science, and 
social studies classes and for home learning decreased in the fourth year. 
Students as a whole reported a minimal level of Core-Content Learning 
(M = 1.95), suggesting that they rarely (a few times a year) or never used 
laptops in core-subject classrooms. Across years, students used laptops in 
their classrooms most often to conduct Internet research, create presenta- 
tions, write with a word processor, and to complete a test or quiz. Students 
also reported a minimal level of laptop use for home learning (M = 1.63) 
indicating that students, on average, used their laptops outside of school 
for homework and learning either not at all or to a trivial extent. 
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Figure 5: 


Level of Implementation (measured on a 0 to 4 scale) 
for three elements of Student Access and Use by 
mean implementation score and year 
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In sum, overall results for the implementation of Technology 
Immersion as measured by standards-based scores showed that the levels 
of support for implementation increased to some extent across project 
years. Similarly, teachers’ reported slightly increased levels of Classroom 
Immersion, although it largely reflected their growing use of technology 
for their own productivity. In contrast, the level of Student Access and Use 
declined in the fourth year. Findings also showed that the level of imple- 
mentation varied by campus. By the end of the fourth year, none of the 
middle schools achieved full immersion, and evidence suggested that just 
a few campuses reached substantial immersion. A majority of schools had 
minimal to partial immersion. 
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Implementation ofTechnology Immersion — 
Implementation Indices 

To further illustrate each school’s level of immersion in the fourth year, 
Table 8 presents the composite campus Implementation Index (z score) 
alongside implementation indices (z scores) for each of the seven compo- 
nents. The Implementation Index is an average score for the Immersion 
Support Index, Classroom Immersion Index, and Student Access and Use 
Index. Z scores have a mean of zero and a standard deviation of 1.0. Thus, 
the campus score indicates how many standard deviations from the mean a 
score lies. Schools with scores above zero have higher values on the compo- 
nents of Technology Immersion, whereas schools with index values below 
zero show less evidence of immersion. Despite some variations in compo- 
nent scores, middle schools with positive values on the Implementation 
Index tended to have component scores that indicated a stronger pres- 
ence of the immersion attributes such as administrative leadership and 
teacher support for immersion. In contrast, middle schools that had the 
most negative values on the Implementation Index generally had negative 
values for nearly all of the immersion components. 
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Table 8: Fourth-Year Implementation ofTechnology Immersion by School 



Immersion Support Index 


Middle 
School (MS) 

Leadership 

Index 

Teacher 

Support 

Index 

Parent/ 

Comm. 

Index 

Technical 

Support 

Index 

PD 

Index 

Classroom 

Immersion 

Index 

Student 
Access/ 
Use Index 

Implementation 

Index 

MSI 

1.69 

2.19 

2.25 

2.05 

0.88 

1.23 

2.02 

2.58 

MS 2 

1.00 

0.36 

0.15 

0.48 

0.54 

1.52 

1.77 

1.78 

MS 3 

1.30 

0.54 

1.60 

1.11 

-0.28 

0.75 

0.49 

1.08 

MS 4 

0.45 

2.27 

0.73 

-0.16 

2.08 

1.71 

-0.99 

0.99 

MS 5 

0.42 

0.50 

0.38 

-0.83 

0.75 

0.10 

0.47 

0.40 

MS 6 

0.01 

0.36 

-1.06 

0.19 

0.52 

-0.13 

0.97 

0.39 

MS 7 

0.49 

-0.56 

-0.09 

0.98 

0.41 

-0.05 

0.25 

0.25 

MS 8 

0.50 

0.13 

0.70 

0.72 

0.01 

-0.45 

0.20 

0.15 

MS 9 

-0.19 

0.35 

-0.11 

-0.74 

0.94 

-0.19 

0.39 

0.12 

MS 10 

-0.53 

0.03 

-0.57 

-1.48 

0.80 

0.45 

-0.33 

-0.17 

MS 11 

-0.83 

0.94 

-1.58 

-1.67 

-1.30 

1.57 

-0.81 

-0.22 

MS 12 

0.81 

-0.23 

-0.88 

-0.17 

-0.71 

0.25 

-0.42 

-0.23 

MS 13 

-0.02 

-0.81 

0.85 

0.95 

-0.02 

0.66 

-1.50 

-0.25 

MS 14 

0.39 

-0.23 

0.43 

-0.02 

0.09 

-0.35 

-0.41 

-0.25 

MS 15 

-1.29 

-0.23 

0.39 

1.08 

0.96 

-1.31 

0.20 

-0.38 

MS 16 

0.85 

-0.10 

-0.50 

-0.13 

-1.77 

-0.77 

0.14 

-0.49 

MS 17 

-1.61 

-1.58 

-0.12 

-0.12 

-1.05 

-1.21 

0.79 

-0.72 

MS 18 

-2.33 

-0.28 

0.81 

0.79 

-2.05 

-2.07 

0.73 

-0.98 

MS 19 

-0.51 

-1.38 

-1.34 

-1.08 

-0.20 

-0.44 

-1.08 

-1.24 

MS 20 

-0.98 

-1.19 

-0.96 

-1.61 

0.04 

-0.45 

-1.21 

-1.33 

MS 21 

0.36 

-1.10 

-1.32 

-0.34 

-0.61 

-0.82 

-1.69 

-1.49 


Note. Implementation indices are z scores with a mean of zero and a standard deviation of 1.0. Scores above 
zero indicate a greater presence ofTechnology Immersion components and higher levels of implementation. 
The Implementation Index is an average score for the Immersion Support Index, Classroom Immersion Index, 
and Student Access and Use Index. 

Findings suggest that the implementation indices are relatively effec- 
tive in discriminating higher and lower implementing schools. Still, there 
are exceptions to the prevailing trends. Some schools, such as MS 3, had 
generally higher implementation values for most of the indicators except 
Professional Development (-0.28). This suggests that professional develop- 
ment for teachers was a lower priority at this school in the fourth year. MS 
4 had generally high levels of school support and Classroom Immersion, 
but students had a low score for Student Access and Use (-0.99) because 
they were not allowed to use their laptops at home for learning. In other 
schools, such as MS 17 and MS 18, students reported higher levels of 


JT-LA 


Evaluating the Implementation Fidelity ofTechnology Immersion 


Shapley, Sheehan, Maloney, & Caranikas-Walker 


33 


technology access and use even though strong implementation supports 
were not in place, and their teachers’ levels of Classroom Immersion were 
low. Results for the Implementation Index combined with evidence from 
standards-based scores suggest that about a quarter of middle schools (6), 
with Implementation Index scores ranging from 0.39 to 2.58 standard 
deviations above the mean, had a stronger presence of the components of 
Technology Immersion compared to other schools, and thus a higher level 
of implementation that more nearly approximated expected standards. 

Table 9 displays the correlations between the seven components of 
Technology Immersion (z scores), with statistically significant coefficients 
denoted in bold. Core-subject teachers’ extent of Classroom Immersion 
was associated at a statistically significant level with their perceptions of 
the strength of the school’s administrative leadership (r = .59), teachers’ 
collective support for technology innovation (r = .67), and the quality of 
professional development (r = .47). Surprisingly, there was virtually no 
association between core-subject teachers’ level of Classroom Immersion 
and Student Access and Use (r = .04). Instead, students’ reported access 
to and use of laptops was positively associated at a statistically significant 
level with teachers’ perceptions of the strength of Parent and Community 
Support (r = .50) and the quality of Technical Support (r = .52) for immer- 
sion. In general, students had more robust technology experiences when all 
teachers in the school supported innovation, technical supports addressed 
maintenance issues that created barriers to technology use, and parents 
and community members supported the school’s technology efforts. 


Table 9: Fourth-Year Correlations ofTechnology Immersion Components 



Immersion Support 


Middle School (MS) 

Leadership 

Teacher 

Support 

Parent 

Support 

Technical 

Support 

PD 

Classroom 

Immersion 

Student 

Access/Use 

Leadership 

1.00 







Teacher Support 

.48* 

1.00 






Parent/Community Support 

.30 

.47* 

1.00 





Technical Support 

.34 

.25 

.76** 

1.00 




Professional Development 

.33 

.47* 

.28 

.09 

1.00 



Classroom Immersion 


0.67** 

.15 

-.03 

.47* 

1.00 


Student Access and Use 

.21 

.38 

.50* 

.52* 

.09 

.04 

1.00 


Note. N = 21 Technology Immersion schools. PD = Professional Development. 
*p < .05. **p<.01. ***p<.001. 
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Association between Implementation and 
Academic Outcomes 

Implementation evidence for the fourth year, similar to previous 
years, revealed wide variation across schools and teachers. Given differ- 
ences in implementation from school-to-school and from classroom-to- 
classroom, we investigated the relationships between implementation 
levels and student academic achievement. For analyses, we used standard- 
ized implementation indicators (z scores) that could be analyzed individu- 
ally or aggregated as component scores. Analyses involved predictors that 
measured school supports for immersion (Immersion Support Index), the 
extent of teachers’ classroom immersion (Classroom Immersion Index), 
and the extent of students’ technology access and use (Student Access and 
Use Index). Our dependent variables were students’ TAKS reading and 
mathematics T scores (standardized scores with a mean of 50 and standard 
deviation of 10). Students included in analyses were continuously enrolled 
in Technology Immersion schools across project years, and thus, about a 
third of students were lost due to attrition. Student retention rates were 
fairly consistent across cohorts and years: Cohort 1 (59%), Cohort 2 (58%, 
61%), and Cohort 3 (69%, 66%). Analyses for student cohorts involved 
approximately 1,000 to 1,400 students. Across cohorts and years, similar 
numbers of core-subject teachers were included in analyses for reading 
(34 to 39 language arts teachers) and mathematics (37 to 40 mathematics 
teachers). 

HLM Analysis 

HLM refers to hierarchical linear models, statistical models used for 
analyzing data in a clustered or nested structure. In education, data struc- 
tures are often hierarchical— that is, lower-level units are nested within 
higher-level units. For example, students are nested within classrooms 
(two levels). HLM models allow researchers to “think about the possible 
mechanisms on each of the levels separately and then join the separate 
models in a joint analysis” (Raudenbush 8t Bryk, 2002, p. xxi). For this 
study, we used a series of two-level HLM models, in which students were 
nested within teachers, to investigate whether the levels of implementa- 
tion for two teacher-related implementation components (Immersion 
Support Index, Classroom Immersion Index) and one student-specific 
component (Student Access and Use Index) were significant predictors of 
students’ TAKS reading and mathematics scores. We used HLM models to 
analyze the effects of implementation on academic achievement for two 
implementation and testing years: (a) third-year implementation, TAKS 

2007, Cohorts 1, 2, and 3; and (b) fourth-year implementation, TAKS 

2008, Cohorts 2 and 3. In sections below, conceptual summaries are pro- 
vided along with HLM equations. 
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Level 1: Student-Level Model 

At the student level, we denote the academic outcome (TAKS reading 
or mathematics achievement) for student i nested in teacher j as Y^. This 
outcome is represented as a function of individual student characteristics 
and model error (r i} ). In Model 1, students’ spring 2007 TAKS T scores were 
regressed on spring 2006 TAKS T scores (or spring 2008 T scores were 
regressed on spring 2007 T scores), Student Access and Use (z score), eco- 
nomic status (0 if not disadvantaged, 1 if disadvantaged), African American 
status (0 if not African American, 1 if African American), Hispanic status 
(0 if not Hispanic, 1 if Hispanic), and gender (0 if male, 1 if female). That 
is, 

Yjj = fS 0 - + ft 1 j(Spring 2006/2007 T score [grand mean centered ]y + 

{*> 2 j(Student Access and Use [grand mean centered J)^ + ft 3 j(Economic 
status )jj + & 4 j(African American status)^ + E> 5 j(Hispanic status )jj + 
ftgjiFemale)^ + r f; -. 

The regression coefficients (K 0; - through fi 6; ) indicate how achievement 
is distributed for teacher j as a function of the measured student charac- 
teristics. 

In Model 2, spring 2007/2008 TAKS T scores were regressed on spring 
2006/2007 TAKS T scores, Laptop Access Days (z score), Core-Content 
Learning (z score), Home Learning (z score), economic status (0 if not dis- 
advantaged, 1 if disadvantaged), African American status (0 if not African 
American, 1 if African American), Hispanic status (0 if not Hispanic, 1 if 
Hispanic), and gender (0 if male, 1 if female). That is, 

Y^ = K 0; - + ft^/Spring 2006/2007 T score [grand mean centered])^ + 
{*> 2 j(Laptop Access Days [grand mean centered])^ + & 3 j(Core-Content 
Learning [grand mean centered])jj + E> 4 j(Home Learning [grand mean 
centered])^ + ft 5 j(Economic status)^ + E> 6 j(African American status)^ + 
E, 7 j(Hispanic status)^ + E> 8 j(Female)jj + r i; -. 

Statistics in Table 10 show that the variance in TAKS reading T scores 
that was between teachers ranged from 10.6% to 25.8%, and the variance in 
TAKS mathematics T scores that was between teachers ranged from 12.0% 
to 33.8%. Thus, in the models described above, the teacher means (K 0; ) 
were specified as randomly varying. The coefficient for spring 2006/2007 
T scores (K 1; ) was also specified as randomly varying (significant reduction 
in model deviance). The coefficients for the remaining independent vari- 
ables in the models were specified as fixed. 
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Table 10: 


Between-Teacher Variance (Intraclass Correlation Coefficients) for 
TAKS Reading and Mathematics Scores 



Cohort 1 

Cohort 2 

Cohort 3 

Grade 8 

Grade 7 

Grade 8 

Grade 6 

Grade 7 

TAKS 

2007 

2007 

2008 

2007 

2008 

Reading 

25.8% 

24.1% 

20.1% 

1 7.2% 

1 0.6% 

Mathematics 

33.8% 

23.0% 

24.4% 

1 2.0% 

1 7.0% 


Note. The total variability of TAKS scores can be divided between variation among students and variation 
among teachers. The percentage of this total variation in TAKS scores between teachers (classrooms) is 
reported here. 


Level 2: Teacher-Level Model 

At the teacher level (Level 2), researchers investigated whether two 
teacher-related implementation components, Immersion Support Index 
(z score) and Classroom Immersion Index (z score), as well as school 
poverty were significant predictors of students’ TAKS reading and math- 
ematics scores (Y^), after adjusting for students’ prior achievement and 
demographic characteristics, and student-level implementation indica- 
tors. School poverty was a continuous variable indicating the percentage 
of economically disadvantaged students in a school, with a mean of 68% 
to 71%. The combined prediction equation for Model 1 is: 

Yy = r 00 + Y 01 (School poverty [grand mean centered]) j + Y 02 (Immersion 
Support [grand mean centered ]) ■ + Y 03 (Classroom Immersion [grand 
mean centered]) j + Y 10 (Spring 2006/2007 T score) + Y 2 o(Student Access 
and Use)^ + Y 30 (Economic status )^ + Y 40 (African American status )^ + 

Y 50 (Hispanic status )^ + Y^CFemale)^ + p 0; + p^fSpring 2006/2007 T 

score)ij + r t j. 

The Model 2 prediction equation is: 

Yjj = Y 00 + Y 01 (School poverty [grand mean centered]) + Y 02 (Immersion 
Support [grand mean centered])j + Y 03 (Classroom Immersion [grand 
mean centered]) j + Y 10 (Spring 2006/2007 T score) ^ + Y 20 (Laptop 
Access Days) { j + Y 30 (Cor e-Content Learning)^ + Y 40 (Home Learning)^ + 
Y S0 (Economic status )^ + Y 60 (African American status)^ + Y 70 (Hispanic 
status )y + Y 80 (Female )^ + u 0; + (Spring 2006/2007 T score ) { j + r 1} . 

In the Level 2 models, Y 01 , Y 02 , and Y 03 represent the impact of the 
teacher-level explanatory variables on the achievement outcome, control- 
ling for all other predictors; and Y 10 , Y 20 , ... Y so represent the impact of 
the student-level explanatory variables on the outcome controlling for all 
other predictors; p 0 j is the unique effect of teacher; on mean achievement 
and Ujj is the unique effect of teacher; on the achievement slope (expected 
change in achievement), after controlling for other predictors. 
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In summary, the student-level models (Level 1) investigated whether 
Student Access and Use (or the elements of Student Access and Use) pre- 
dicted higher 2007 or 2008 TAKS scores, after adjusting for initial achieve- 
ment, student demographic characteristics, school poverty, Immersion 
Support, and Classroom Immersion. The teacher-level models (Level 2) 
investigated whether the Immersion Support Index (average school z 
score) and Classroom Immersion Index (language arts or mathematics 
teacher’s individual z score) predicted higher 2007 or 2008 TAKS scores, 
after adjusting for school poverty, students’ prior achievement and demo- 
graphic characteristics, and Student Access and Use. 3 

TAKS Reading 

Estimates of the effects of implementation on Cohorts 1, 2, and 3 stu- 
dents’ TAKS reading T scores are presented in Table 11. At the teacher 
level in Model 1, we investigated whether the strength of language arts 
teachers’ school support for implementation (Immersion Support) and 
their reported levels of Classroom Immersion were predictors of students’ 
reading achievement. Results revealed that only one of the teacher-level 
implementation measures was a statistically significant predictor of TAKS 
reading scores. After controlling for student variables (prior achievement, 
demographic characteristics, Student Access and Use) and other teacher 
variables (school poverty and Classroom Immersion), Immersion Support 
was a positive predictor of Cohort 1 eighth graders’ reading achievement. 
Moreover, language arts teachers’ level of Classroom Immersion was an 
inconsistent predictor of students’ TAKS reading achievement. After 
adjusting for other variables in the analysis, Cohort 2 students, who had 
language arts teachers with average levels of Classroom Immersion, had 
slightly higher TAKS reading T scores (0.69 and 0.22 points, respectively) 
than students whose teachers had below average Classroom Immersion 
scores. Conversely, Cohorts 1 and 3 students who had language arts 
teachers with average levels of Classroom Immersion had lower TAKS 
scores (-0.23, -0.71, and -0.21 T- score points) than students whose lan- 
guage arts teachers had below average Classroom Immersion. 
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Table 1 1 : Hierarchical Regression Models Predicting the Effects 

ofTechnology Immersion Implementation Indicators on 
TAKS Reading Achievement 



Cohort 1 

Cohort 2 

Cohort 3 

Eighth Graders 
2007, N = 1,217 

Seventh Graders 
2007, N= 1,297 

Eighth Graders 
2008, N= 1,101 

Sixth Graders 
2007, N = 1,606 

Seventh Graders 
2008, N = 1,168 

Gamma 

Coeff. 

t-value 

Gamma 

Coeff. 

f-value 

Gamma 

Coeff. 

r-value 

Gamma 

Coeff. 

f- value 

Gamma 

Coeff. 

r-value 

Model 1 

Intercept 

49.803 

128.06*** 

50.984 

74.56*** 

50.264 

128.56*** 

48.869 

103.51*** 

49.204 

99.61*** 

Teacher-level predictors 

School 

poverty 

3.112 

1.76 

3.125 

0.97 

-0.016 

-1.18 

1.185 

0.75 

-0.021 

-1.15 

Immersion 

Support 

1.340 

5.62*** 

-0.129 

-0.25 

0.064 

0.52 

-0.030 

-0.10 

-0.315 

-1.16 

Classroom 

Immersion 

-0.234 

-0.64 

0.688 

1.21 

0.215 

1.45 

-0.705 

-2.02t 

-0.211 

0.73 

Student-level predictors 

Spring 
2006/2007 
T score 

0.537 

23.03*** 

0.654 

19.77*** 

0.689 

21.38*** 

0.532 

27.99*** 

0.666 

34.78*** 

Student 
Access 
and Use 

0.542 

2.05* 

0.895 

2.56* 

0.466 

1.38 

0.523 

1.88t 

0.791 

1.93t 

Eco. Disad- 
vantaged 

-1.039 

-2.53* 

-0.625 

-1.13 

-0.601 

-1.33 

-0.266 

-0.68 

-0.875 

-1.63 

African 

American 

-0.285 

-0.31 

-2.562 

-2.91** 

-1.862 

-2.12* 

-0.837 

-1.21 

-1.649 

-2.18* 

Hispanic 

-0.949 

-2.07* 

-2.443 

-2.84** 

-0.164 

-0.43 

-0.939 

-1.91 1 

-0.518 

-0.83 

Female 

0.394 

1.10 

-0.027 

-0.06 

0.674 

1.95t 

0.785 

2.98** 

0.216 

0.52 


(Continued) 
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Table 1 1 : Hierarchical Regression Models Predicting the Effects 

ofTechnology Immersion Implementation Indicators on 
TAKS Reading Achievement (continued) 



Cohort 1 

Cohort 2 

Cohort 3 

Eighth Graders 
2007, N= 1,21 7 

Seventh Graders 
2007, N= 1,297 

Eighth Graders 
2008, N= 1,101 

Sixth Graders 
2007, N = 1,606 

Seventh Graders 
2008, N = 1,168 

Gamma 

Coeff. 

t-value 

Gamma 

Coeff. 

Lvalue 

Gamma 

Coeff. 

Lvalue 

Gamma 

Coeff. 

L- value 

Gamma 

Coeff. 

Lvalue 

Model 2 

Intercept 

49.803 

130.90*** 

51.196 

75.77*** 

50.326 

126.61*** 

48.883 

104.32*** 

49.274 

100.01*** 

Teacher-level predictors 

School 

poverty 

3.287 

1.65 

2.916 

0.93 

-0.019 

-1.32 

1.026 

0.65 

-0.032 

-1.88 

Immersion 

Support 

1.325 

5.45*** 

0.143 

0.32 

0.105 

0.85 

-0.030 

-0.10 

-0.263 

-0.98 

Classroom 

Immersion 

-0.261 

-0.72 

0.762 

1.33 

0.239 

1.55 

-0.688 

-2.03* 

0.365 

1.41 

Student-level predictors 

Spring 
2006/2007 
T score 

0.537 

22.03*** 

0.640 

18.44*** 

0.686 

20.83*** 

0.531 

27.80*** 

0.658 

34.87*** 

Laptop 

Access 

Days 

-0.033 

-0.32 

0.120 

0.41 

0.195 

0.89 

0.060 

0.22 

-0.186 

-0.73 

Core- 

Content 

Learning 

0.246 

0.81 

-0.485 

-2.00* 

-0.098 

-0.41 

-0.050 

-0.30 

-0.344 

-1.54 

Home 

Learning 

0.311 

1.37 

1.010 

6.14*** 

0.304 

1.27 

0.394 

3.01** 

0.985 

4.77*** 

Eco. Disad- 
vantaged 

-1.054 

-2.60* 

-0.558 

-1.06 

-0.627 

-1.39 

-0.289 

-0.74 

-0.765 

-1.41 

African 

American 

-0.284 

-0.31 

-2.423 

-2.84** 

-1.872 

-2.12* 

-0.785 

-1.16 

-1.624 

-2.27* 

Hispanic 

-0.931 

-1.96* 

-2.630 

-3.05** 

-0.202 

-0.52 

-0.904 

-1.86t 

-0.648 

-1.07 

Female 

0.415 

1.19 

-0.142 

-0.32 

0.641 

1.88 

0.752 

2.82** 

0.152 

0.37 


Note.TAKS 2007 = third implementation year; TAKS 2008 = fourth implementation year. 

Number of language arts teachers 2007: Cohort 1 = 39, Cohort 2 = 37, Cohort 3 = 41. 

Number of language arts teachers 2008: Cohort 2 = 37, Cohort 3 = 34. 
tp < .1 0. *p < .05. **p < .01 . ***p < .001 . 

In contrast to teacher-level predictors, the level of Student Access 
and Use (of technology) was a stronger and more consistent predictor of 
reading achievement. After controlling for students’ prior reading achieve- 
ment, demographic characteristics, and teacher-level variables (school 
poverty and implementation components), the Student Access and Use 
effect on TAKS reading achievement was consistently positive for Cohort 
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1 (0.54 T - score point), Cohort 2 (0.90 and 0.47 points), and Cohort 3 (0.52 
and 0.79 points). Results generally were either statistically significant or 
marginally significant. 

Additionally, we conceptualized Student Access and Use as having mul- 
tiple elements (Laptop Access Days, Core-Content Learning, and Home 
Learning), and thus, were interested in separately predicting variation for 
each element. Table 11 provides statistics for the HLM models used to 
predict each of the three elements (Model 2). Findings revealed that Home 
Learning — which measured the extent of a student’s laptop use outside 
of school for homework in each of the four core-subject areas and for 
learning games — was the strongest implementation predictor of reading 
achievement. The Home Learning effect on TAKS reading scores was posi- 
tive for Cohort 1 (0.31 T- score point), Cohort 2 (1.01 and 0.30 points), and 
Cohort 3 (0.39 and 0.99 points). Results were statistically significant for 
sixth graders (Cohort 3) and seventh graders (both Cohorts 2 and 3). As an 
example, after controlling for all of the other variables in the analysis, an 
economically advantaged, non-minority, male seventh grader (Cohort 3) 
with a score one standard deviation above average for Home Learning (z = 
1.00), had a 0.99 T - score point higher TAKS reading score. Moreover, with 
each additional standard deviation increase in Home Learning, students’ 
reading achievement was predicted to increase. 

In contrast to Home Learning, the number of days during the school 
year that students had laptops available for use (Laptop Access Days) was 
not a statistically significant predictor of students’ reading achievement. 
The frequency that students reported using their laptops across their four 
core-subject classes (Core-Content Learning) was a positive predictor of 
reading achievement for Cohort 1 (although not by a statistically signifi- 
cant margin), but a negative predictor of reading achievement for Cohorts 

2 and 3 students, after controlling for other variables in the analysis. 

TAKS Mathematics 

We also estimated the effects of implementation on students’ TAKS 
mathematics T scores. Like reading, we examined implementation effects 
for students and teachers (see Table 12). Comparable to reading, in Model 1, 
the school- and teacher-level implementation indicators were typically not 
statistically significant predictors of students’ TAKS mathematics scores. 
After controlling for other variables in the analysis, Immersion Support 
was a positive predictor for Cohort 1 students’ mathematics achievement 
but a mixed predictor for Cohorts 2 and 3 students’ mathematics scores. 
Similarly, after statistical adjustments for the other variables in the anal- 
ysis, mathematics teachers’ reported Classroom Immersion level was an 
inconsistent predictor of TAKS math achievement for Cohorts 1, 2, and 3 
students. 
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Table 1 2: Hierarchical Regression Models Predicting the Effects 

ofTechnology Immersion Implementation Indicators on 
TAKS Mathematics Achievement 



Cohort 1 

Cohort 2 

Cohort 3 

Eighth Graders 
2007, N= 1,1 74 

Seventh Graders 
2007, N = 1,382 

Eighth Graders 
2008, N = 999 

Sixth Graders 
2007, N = 1,389 

Seventh Graders 
2008, N= 1,165 

Gamma 

Coeff. 

f- value 

Gamma 

Coeff. 

t-value 

Gamma 

Coeff. 

t-value 

Gamma 

Coeff. 

t-value 

Gamma 

Coeff. 

t-value 

Model 1 

Intercept 

51.550 

83.22*** 

48.703 

85.70*** 

50.542 

108.15*** 

47.917 

80.95*** 

50.313 

101.20*** 

Teacher-level predictors 

School 

poverty 

8.713 

2.84** 

4.608 

1.55 

0.011 

0.57 

5.418 

1.55 

-0.010 

-0.40 

Immersion 

Support 

1.464 

2.99** 

0.643 

0.98 

-0.168 

-0.55 

-0.206 

-0.26 

0.026 

0.07 

Classroom 

Immersion 

-0.174 

-0.23 

0.621 

1.76t 

0.451 

1.03 

0.292 

0.59 

-0.614 

-1.49 

Student-level predictors 

Spring 
2006/2007 
T score 

0.707 

28.19*** 

0.731 

28.15*** 

0.702 

32.65*** 

0.689 

32.23*** 

0.730 

35.24*** 

Student 
Access 
and Use 

0.809 

3.34** 

0.864 

3.40** 

0.303 

1.01 

0.889 

2.97** 

0.505 

1.39 

Eco. Disad- 
vantaged 

-0.736 

-1.96* 

0.129 

0.31 

-0.607 

-1.41 

-0.518 

-0.94 

-0.632 

-1.26 

African 

American 

-1.037 

-1.63 

-0.553 

-1.11 

-1.528 

-2.19* 

-1.219 

-1.34 

-2.475 

-2.74*** 

Hispanic 

-0.611 

-0.82 

-1.187 

-2.31* 

-0.436 

-1.08 

-0.667 

-0.98 

-0.998 

-1.73t 

Female 

-0.477 

-1.52 

-0.178 

-0.60 

-0.338 

-0.86 

1.025 

2.67** 

0.509 

1.78t 


(Continued) 
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Table 1 2: Hierarchical Regression Models Predicting the Effects 

ofTechnology Immersion Implementation Indicators on 
TAKS Mathematics Achievement (continued) 



Cohort 1 

Cohort 2 

Cohort 3 

Eighth Graders 
2007, N = 1,1 74 

Seventh Graders 
2007, N = 1,382 

Eighth Graders 
2008, N = 999 

Sixth Graders 
2007, N = 1,389 

Seventh Graders 
2008, N= 1,165 

Gamma 

Coeff. 

t-value 

Gamma 

Coeff. 

Lvalue 

Gamma 

Coeff. 

t-value 

Gamma 

Coeff. 

L- value 

Gamma 

Coeff. 

Lvalue 

Model 2 

Intercept 

51.612 

85.74*** 

48.787 

85.55*** 

50.588 

109.51*** 

47.912 

79.65*** 

50.343 

98.36*** 

Teacher-level predictors 

School 

poverty 

8.304 

2.82** 

4.617 

1.61 

0.007 

0.35 

5.191 

1.49 

-0.013 

-0.51 

Immersion 

Support 

1.432 

3.06** 

0.703 

1.09 

-0.159 

-0.52 

-0.186 

-0.24 

0.214 

0.57 

Classroom 

Immersion 

-0.244 

-0.34 

0.612 

1.80t 

0.416 

0.98 

0.340 

0.70 

-0.584 

-1.42 

Student-level predictors 

Spring 
2006/2007 
T score 

0.699 

27.99*** 

0.728 

27.24*** 

0.698 

33.54*** 

0.686 

31.42*** 

0.725 

35.57*** 

Laptop 

Access 

Days 

-0.002 

-0.01 

0.244 

1.81 

0.019 

0.12 

0.278 

1.02 

0.181 

0.72 

Core- 

Content 

Learning 

0.019 

0.07 

0.032 

0.15 

-0.146 

-0.72 

0.057 

0.29 

-0.322 

-1.70+ 

Home 

Learning 

0.675 

3.52** 

0.508 

2.54* 

0.324 

1.74+ 

0.504 

2.68** 

0.482 

2.07* 

Eco. Disad- 
vantaged 

-0.801 

-2.12* 

0.140 

0.33 

-0.627 

-1.48 

-0.506 

0.91 

-0.605 

-1.19 

African 

American 

-1.061 

-1.68 

-0.539 

-1.08 

-1.542 

-2.23* 

-1.169 

-1.27 

-2.418 

-2.70** 

Hispanic 

-0.562 

-0.78 

-1.249 

-2.42* 

-0.442 

-1.10 

-0.647 

-0.94 

-1.017 

-1.78+ 

Female 

-0.490 

-1.54 

-0.215 

-0.69 

-0.376 

-0.98 

0.974 

2.49* 

0.438 

1.48 


Note.TAKS 2007 = third implementation year; TAKS 2008 = fourth implementation year. 
Number of mathematics teachers 2007: Cohort 1 = 39, Cohort 2 = 40, Cohort 3 = 33. 
Number of mathematics teachers 2008: Cohort 2 = 37, Cohort 3 = 38. 
tp < .1 0. *p < .05. **p < .01 . ***p < .001 . 
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Similar to reading achievement, Student Access and Use was a con- 
sistently positive predictor of students’ TAKS mathematics scores. 
Controlling for students’ prior math achievement, demographic charac- 
teristics, and teacher-level variables (implementation components as well 
as school poverty), the sizes of the Student Access and Use effects were 
statistically significant for 2007 TAKS scores associated with third-year 
implementation (0.81, 0.86, and 0.89 T- score points for Cohorts 1, 2, and 
3 students, respectively). Student Access and Use was a positive although 
not statistically significant predictor of 2008 TAKS mathematics scores 
associated with fourth-year implementation. 

Comparable to TAKS reading achievement, we wanted to have a better 
understanding of the association between students’ reported technology 
access and use and mathematics test scores. Thus, we used an HLM model 
to predict mathematics achievement for each of the three Student Access 
and Use elements (Laptop Access Days, Core-Content Learning, and Home 
Learning). Results for Model 2 presented in Table 12 show that the extent 
to which students reported using their laptops for Home Learning was a 
positive and statistically significant predictor of TAKS mathematics scores. 
As an example, after controlling for the other variables, an economically 
advantaged, non-minority, male eighth grader in Cohort 1 with a Home 
Learning score about one standard deviation above average (z = 0.99) had 
a 0.68 T - score point higher TAKS mathematics score. With each additional 
standard deviation increase in Home Learning, students’ mathematics 
achievement was predicted to increase incrementally. 


Discussion 

This study advances research on one-to-one computing environments 
by defining a comprehensive Technology Immersion model (i.e., a laptop 
computer for every student and teacher, wireless Internet access, cur- 
ricular and assessment resources, professional development, and ongoing 
technical and pedagogical support), positing a theoretical framework 
for how the model is expected to work, and examining how implemen- 
tation indicators link to student outcomes. The Technology Immersion 
model assumes that school-wide provisions of technology resources and 
supports will produce teachers who are more technologically adept, use 
laptops and digital resources to transform their teaching, and have stu- 
dents use technology more often in their classrooms. Improved school 
and classroom environments, in turn, should allow students to have more 
engaging schoolwork, and laptops should extend student learning beyond 
school walls and the school day. Changes in students’ learning experiences, 
accordingly, are expected to improve academic achievement as measured 
by state assessments. In the sections to follow, we discuss how Technology 
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Immersion was implemented at 21 pilot schools, factors that influenced 
progress, and associations with student achievement. In some instances, 
our explanations reflect the views expressed by school administrators, 
teachers, and students during interviews and focus groups conducted as 
part of school site visits near the end of the fourth implementation year 
(Shapley, Sheehan, Maloney, & Caranikas-Walker, 2009). Additionally, 
we discuss the implications of our findings for other educators who are 
undertaking one-to-one laptop initiatives. 

Implementation ofTechnology Immersion 

Full implementation of the Technology Immersion model was challenging, 
with just 6 of 21 schools reaching substantial levels of implementation hy the 
end of the fourth year. The Technology Immersion model was expected to 
overcome obstacles that in the past have posed barriers to effective tech- 
nology use in schools by promoting robust access to technology resources 
in concert with substantive supports from school leaders, teachers, par- 
ents, technicians, and professional development providers. Mean immer- 
sion standard scores, measured on a 4-stage progression encompassing 
minimal (0 to 1.99), partial (2.00 to 2.99), substantial (3.00 to 3.49), and 
full immersion (3.50 to 4.00) levels, revealed small yearly increases across 
most of the implementation support components (Leadership, Teacher 
Support, Parent/Community Support, and Professional Development) as 
well as modest increases in teachers’ overall level of Classroom Immersion. 
Unexpectedly, the level of Student Access and Use declined slightly across 
three years (from 2.17 to 2.07). An examination of the correlations among 
fourth-year implementation measures revealed notable differences and 
similarities in the kinds of supports that advanced teachers’ or students’ 
technology use. For core-subject teachers, higher levels of Classroom 
Immersion were associated with the strength of administrative leadership, 
teachers’ collective support for innovative practices, and the quality of 
professional development. Students, on the other hand, had more robust 
access to and use of technology when all teachers in the school supported 
technology innovation, support from technicians addressed maintenance 
issues that created obstacles to laptop use, and parents and community 
members supported the school’s one-to-one laptop initiative. A common 
element was the importance of support from all teachers in the school. 
When teachers at a school shared understandings about the use of tech- 
nology for learning and were supportive of technology integration, imple- 
mentation was stronger at both the classroom and student levels. 
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Core-subject teachers at 4 of 21 schools achieved substantial levels of 
Classroom Immersion in the fourth year, whereas teachers at 16 schools 
had partial implementation levels and teachers at one school had minimal 
Classroom Immersion. As a whole, the standards-based implementation 
scores for Classroom Immersion increased slightly across years (from 2.48 
to 2.69). Teachers’ use of technology for their own purposes (Professional 
Productivity) increased the most (2.73 to 3.04), while the frequency with 
which teachers allowed students in their classes to use technology for 
learning (Student Activities) remained relatively stable across years (from 
2.33 to 2.43). Thus, teachers, on average, used technology increasingly to 
support their own teaching but there was little change in the frequency of 
students’ technology use in classes. Nevertheless, some teachers at almost 
every school embraced technology and had higher Classroom Immersion 
ratings, but teachers reached higher Classroom Immersion levels as a 
group if they worked in schools with the most supportive environments. 

Students’ access to laptops and use of laptops for learning fell short of 
expectations. By the end of the fourth year, there were no schools that had 
reached substantial to full levels of Student Access and Use, with all stu- 
dents having consistent access to laptops, using laptops almost daily in 
core-subject classes, and using laptops outside of school for homework 
and learning. Moreover, the percentages of schools with at least partial 
levels of Student Access and Use decreased across evaluation years (76%, 
68%, and 57%), while the percentages of schools with minimal student 
access and use increased (24%, 32%, and 43%). Several factors affected 
students’ opportunities to use their laptops, including mainly time lost for 
repairs due to aging laptops, schools that opted to transfer laptops from 
individual students to carts or classroom sets, schools that restricted stu- 
dents’ use of laptops outside of school, and teachers’ preferences regarding 
classroom laptop use. Putting laptops on carts or distributing laptops as 
classroom sets appeared to provide more consistent student access to 
laptops (although not “ownership”) at some schools in the fourth year — 
however, such configurations did not increase the frequency of students’ 
classroom activities with laptops, and taking laptops out of the hands of 
individual students either reduced or eliminated their opportunities to 
use laptops outside of school for learning. 

The lack of a start-up year for planning was a major barrier to effective 
implementation. The majority of middle schools received their Technology 
Immersion Pilot (TIP) grant awards just before the start of the first imple- 
mentation year. Thus, many administrators and teachers in spring 2008 
thought implementation would have progressed more smoothly if there 
had been a start-up year to plan for immersion. Various respondents said a 
planning year would have allowed them to involve teachers in the decision 
to become an “immersed” school, to develop a plan for managing laptops 
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(especially at larger campuses with as many as 1,500 laptops), to build the 
school’s infrastructure for wireless technology, to provide professional 
development for teachers to strengthen their technical skills and ability to 
plan technology-integrated lessons, and to give teachers a chance to “try 
out” lessons with laptops in the classroom before students had their own 
laptops. 

Respondents at higher implementing schools reported that committed 
leaders, thorough planning, teacher huy-in, preliminary professional develop- 
ment for teachers, and a commitment to the transformation of student learning 
were keys to their successful implementation of Technology Immersion. 
Respondents at middle schools that had been more successful attributed 
effective implementation to several factors. Foremost, despite a quick start, 
district and school administrators had a well-conceived plan for imple- 
mentation, were excited about the project, and listened to teacher input. 
Administrators had high expectations for technology use but allowed time 
for teachers to become comfortable. One teacher explained: 

We had the right combination of encouragement and push... 

Leadership, encouragement, and push. It wasn’t punitive, it was 

positive... but they kept up the pressure. ..That constant, positive 

pressure moved me forward. 

Professional development for teachers was a high priority. Training 
typically began before the first year started and was ongoing across 
implementation years. These schools also had collegial cultures. Teachers 
learned by “seeing what other teachers were doing and how they were 
implementing technology.” “We were all in this together,” explained one 
teacher. The improvement of students’ learning experiences was a driving 
force for higher quality implementation at these schools. Despite myriad 
laptop management issues, respondents believed the challenges had been 
worthwhile because one-to-one student laptops and digital resources had 
increased the depth of learning across subject areas, exposed students to 
more real-life experiences, and allowed students to demonstrate greater 
responsibility. Conversely, many respondents at lower implementing 
schools reported that administrative turnover, noncommittal teachers, 
insufficient professional development, inadequate school infrastructures, 
and laptop management problems were impediments to effective imple- 
mentation of the Technology Immersion model. 

The implementation trends reported for Technology Immersion are 
generally consistent with other research on whole-school reform and 
one-to-one initiatives. Specifically, higher levels of implementation were 
associated with administrative leaders who set the direction for change, 
developed supportive policies, fostered collaborative school cultures, and 
acquired resources (e.g., Bradburn & Osborne, 2007; Kurki et al., 2005; 
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Vernez et al., 2006; Zucker & McGhee, 2005). Effective leaders recognized 
the critical importance of teachers’ commitment to change (Vernez et al., 
2006), and the need to influence their beliefs about technology’s poten- 
tial for effective teaching and learning (Penuel, 2006; Russell et al., 2002; 
Windschitl & Sahl, 2002). Consistent with other studies, schools had diffi- 
culty changing instructional practices (Vernez et al., 2006). Many teachers 
at immersion schools, especially veterans, opted to continue traditional 
practices and rejected practices that required innovation and instructional 
change (Cuban, 2001; Russell et al., 2004). Teachers who made greater 
progress participated in high-quality sustained professional development 
that built their technology skills and understanding of curricular integra- 
tion, and they had follow-up support as they implemented new practices 
in their classes (Bradburn & Osborne, 2007; Lowther et al., 2003; Neugent 
& Fox, 2007; Owen et al., 2005-06). 

Despite efforts aimed at promoting a comprehensive reform model, 
many Technology Immersion schools, like CSR schools, tended to imple- 
ment model components selectively (Vernez et al., 2006). Technology 
Immersion schools were most likely to modify the requirement that stu- 
dents have laptops available for use 24/7 (24 hours a day, 7 days a week). 
Decisions sometimes reflected educators’ financial or security concerns, 
but restrictions also arose from some educators’ beliefs that middle-school 
students were too immature to assume responsibility for expensive laptops 
outside of school or that students would not use their laptops productively 
outside of classrooms. 

Association between Implementation and 
Student Achievement 

The ultimate goal of immersing middle schools in technology was to 
increase students’ academic achievement as measured by state assess- 
ments. Knowing that prior research has linked implementation quality with 
positive effects on students’ achievement (Borman et al., 2003; Borman, 
2005), and recognizing the implementation variations across Technology 
Immersion schools and classrooms, it was important to understand 
whether stronger implementation of Technology Immersion was associ- 
ated with higher test scores. We limited our analysis to students’ reading 
and mathematics TAKS scores because students completed tests for those 
subjects each year, and thus, pre-tests were available to control for prior 
achievement. Our analysis aimed to assess the predictive strength of vari- 
ables that measured the extent to which the Technology Immersion model 
was implemented as designed at the school, teacher, and student levels. 
We used two-level HLM models with individual students nested within 
their language arts and mathematics teachers to test the “value added” to 
student achievement by teachers’ Immersion Support (a school-level mea- 
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sure reflecting all teachers) and language arts and mathematics teachers’ 
Classroom Immersion (individual teacher measure), and to test the “value 
added” to academic achievement by Student Access and Use (individual 
student measure) and three elements of access and use (Laptop Access 
Days, Core-Content Learning, and Home Learning). Evidence from the 
third and fourth implementation years allowed researchers to examine 
relationships for schools that had sufficient time to reach more advanced 
stages of implementation and to examine the replication of findings across 
years. 

The implementation strength of Student Access and Use (of technology) 
was a consistently positive predictor of students’ TAKS reading and mathe- 
matics scores. In our HLM models (Model 1), Student Access and Use was 
an aggregate implementation measure of the extent to which a student 
had access to a laptop throughout the school year (number of days), the 
frequency of technology use for learning in core-content classes, and the 
extent of laptop use for homework and learning games. Student-level 
results showed that after controlling for a student’s prior achievement 
and demographic characteristics and other implementation variables, the 
composite measure of Student Access and Use was a consistently positive 
predictor of students’ TAKS reading scores for Cohorts 1, 2, and 3 students 
across two implementation years. Gamma coefficients generally were 
either statistically significant or marginally significant. Similar to reading, 
Student Access and Use was a consistently positive predictor of students’ 
TAKS mathematics scores. Gamma coefficients for the third implementa- 
tion year were statistically significant for Cohorts 1, 2, and 3 students, but 
the coefficients for the fourth implementation year were not statistically 
significant. 

Students’ use of their laptops for Home Learning — a measure of the extent 
to which a student used a laptop outside of school for homework in the four 
core-subject areas or for learning games — was the strongest implementation 
predictor of students’ TAKS reading and mathematics scores. Additional HLM 
models (Model 2) examined the extent to which each of the elements of 
Student Access and Use predicted student achievement, after controlling 
for other variables in the analysis. Of the three elements of Student Access 
and Use, a student’s use of a laptop for Home Learning (i.e., use of a laptop 
outside of school for homework in core-subject areas and for learning 
games) was the strongest implementation predictor of both TAKS reading 
and mathematics achievement. Home Learning was a positive predictor 
of TAKS reading scores for Cohorts 1, 2, and 3 students. Results were sta- 
tistically significant for sixth and seventh graders but not eighth graders. 
The extent of laptop use for Home Learning was a positive and statistically 
significant predictor of TAKS mathematics scores for Cohorts 1, 2, and 3 
students. In contrast, a student’s reported number of Laptop Access Days 
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and the frequency that a student reported using a laptop for Core-Content 
Learning were mixed predictors (positive and negative) and were typically 
not statistically significant predictors of test scores. 

It should be noted, however, that about a third of students were lost 
through attrition. Consequently, reported results represent outcomes 
for students who were continuously enrolled in middle schools and find- 
ings may not generalize to the more mobile group of students who left 
schools. Nevertheless, students who remained in schools, like the full 
student population, were largely economically disadvantaged and mem- 
bers of minority groups. Thus, the finding for Home Learning is impor- 
tant because it underscores the role that individual student laptops play in 
promoting ubiquitous learning and equalizing the out-of-school learning 
opportunities for students in disadvantaged family and school situations 
(Burbules, 2007; Dede, 2007). Individual student laptops, in contrast to 
laptops on carts or computers available in libraries, labs, and classrooms, 
expanded where and how student learning occurred. Students at higher 
Technology Immersion schools typically had access to their laptops “24/7,” 
and teachers at these schools encouraged students’ use of laptops outside 
of school by engaging students in projects or assignments in class that 
motivated students to continue working outside of school. Also, access to 
electronic resources and textbooks on laptops motivated many students to 
continue working on assignments at home (Shapley, Maloney, Caranikas- 
Walker, & Sheehan, 2008). 

The school’s level of Immersion Support and teachers’ reported levels of 
Classroom Immersion were inconsistent predictors of students’ TAKS reading 
and mathematics scores. After controlling for student-level variables and 
other teacher-level variables in analyses, the school’s level of Immersion 
Support (a composite measure of implementation supports) was an incon- 
sistent predictor of student achievement. Similarly, the levels of Classroom 
Immersion reported by language arts teachers were inconsistent, and for 
the most part, not statistically significant predictors of students’ TAKS 
reading scores. Likewise, the levels of Classroom Immersion reported by 
mathematics teachers were inconsistent and not statistically significant 
predictors of students’ TAKS mathematics scores. There are several poten- 
tial explanations for teacher-level findings. First, measures of Immersion 
Support may have an indirect rather than a direct association with student 
achievement through the provision of supports for changes in teachers’ 
instructional practices and students’ learning experiences. Whole-school 
supports also may have a stronger relationship with composite measures 
of student achievement (e.g., percentages of students passing all TAKS 
tests) rather than students’ test scores for individual subject areas. 
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At the individual teacher level, the measured elements of Classroom 
Immersion were designed to capture teachers’ growth in ideological beliefs 
(Technology Integration, Learner-Centered Instruction), the frequency of 
classroom activities with technology (Student Activities), the use of tech- 
nology to extend classroom boundaries (Communication), and the use of 
technology for administrative and instructional purposes (Professional 
Productivity). Scores for individual variables and the composite Classroom 
Immersion Index were useful in measuring teachers’ development in areas 
that have been associated with positive effects on students’ learning expe- 
riences (Shapley et al., 2009). However, there may be other technology- 
related variables that link more directly to student academic achievement. 
For example, some researchers recommend the use of specific measures 
of technology use rather than more general measures of how often tech- 
nology is used (Bebell, Russell, & O’Dwyer, 2004; O’Dwyer, Russell, Bebell, 
& Tucker-Seeley, 2005; O’Dwyer, Russell, Bebell, & Seeley, 2008). Thus, 
future exploratory analyses might test the associations between students’ 
test scores and the specific technology uses reported by teachers or stu- 
dents (e.g., students express themselves in writing; enter, calculate, and 
graph information; visually represent or investigate concepts). Student- 
reported activities with technology in classrooms will likely provide the 
most salient predictors of academic effects because students’ experiences 
varied within classrooms due to inconsistent access to laptops caused by 
lost days for repairs, disciplinary infractions, or other reasons. 

Alternatively, it is also possible that the quality of Classroom Immersion 
in language arts and mathematics classes ( partial immersion, on average) 
or the frequency of student laptop use in core-content classes ( rarely or 
never used, on average) have not reached levels that are effective enough to 
impact student achievement. Additionally, immersion effects on achieve- 
ment may reflect the interdisciplinary efforts of many teachers rather 
than single core-subject teachers. For instance, the use of laptops for social 
studies research and compositions may positively affect reading outcomes. 
Similarly, exposure to investigations and problem-solving activities in sci- 
ence may positively affect mathematics scores. 

In conclusion, findings from four implementation years suggest that 
Technology Immersion can be implemented with fidelity. If districts and 
schools are committed to the model’s specifications, especially students’ 
personal access to laptops within and outside of school, the prospects for 
raising academic achievement are promising. Certainly, effective tech- 
nology integration involved much more than just buying laptops for stu- 
dents. Technology Immersion requires a comprehensive approach that 
transforms the school culture, changes the nature of teaching and learning, 
and expands the educational boundaries of the school and classrooms. This 
study confirms that fundamental school change is difficult and requires 
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a long-term commitment at all levels of the school system (board mem- 
bers, superintendent, principals, teachers, students, and parents). Given 
the challenges and costs of implementing Technology Immersion, state- 
wide implementation of the model may not be feasible. However, those 
districts and schools that are committed to Technology Immersion should 
have federal and state support for their innovative school-reform efforts. 
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Appendix A 

Technology Immersion Components 

The Texas Education Agency selected three lead vendors as providers 
ofTechnology Immersion packages (Dell Computer, Inc., Apple Computer 
Inc., and Region 1 Education Service Center [ESC]). Sections to follow pro- 
vide descriptions of package components. 

Wireless laptops and productivity software 

All vendors offered a wireless laptop as the mobile computing device. 
Campuses could select either Apple laptops (iBook and MAC OSX) or 
Dell laptops (Inspiron or Latitude with Windows OS). For Apple laptops, 
AppleWorks provides a suite of productivity tools, including Keynote pre- 
sentation software, Internet Explorer, Apple Mail, iCal calendars, iChat 
instant messaging, and iLife Digital Media Suite (iMovie, iPhoto, iTunes, 
GarageBand, and iDVD). For Dell laptops, Microsoft Office includes Word, 
Excel, Outlook, PowerPoint, and Access. In addition, eChalk serves as a 
“portal” to other web-based applications and resources included in the 
immersion package and a student-safe e-mail solution. Region 1 ESC pro- 
vided Dell products. 

Online instructional and assessment resources 

Immersion packages included a variety of digital resources. Apple 
included the following online resources: netTrekker (an academic Internet 
search engine), Beyond Books from Apex Learning (reading, science, and 
social studies online), ClassTools Math from Apex Learning (complete 
math instruction), Explor eLearning Math and Science (supplemental math/ 
science curriculum), TeenBiz3000 from Achieve 3000 (differentiated 
reading instruction), and My Access Writing from Vantage Learning (sup- 
port for writing proficiency). Dell, Inc. selected netTrekker (an academic 
Internet search engine) and Connected Tech from Classroom Connect 
(technology-based lessons and projects). Region 1 ESC selected Connected 
Tech but also added a variety of teaching and learning resources including 
Unitedstreaming (digital videos), Encyclopedia Britannica, EBSCO (data- 
bases), NewsBank, and K12 Teaching and Learning Center. For the Apple 
package, AssessmentMaster (Renaissance Learning) provides a formative 
assessment in all four core subject areas. Both the Dell and Region 1 ESC 
packages provide i-Know (CTB McGraw Hill) for core-subject assessment. 
In addition, all campuses have access to the online Texas Mathematics 
Diagnostic System (TMDS) and Texas Science Diagnostic System (TSDS) 
that are provided free of charge by the state. 
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Professional development 

Each immersion package includes a different professional development 
provider. Apple uses its own professional development model, whereas the 
Dell package relies on Pearson Learning Group, a commercial provider (for- 
merly Co-nect), to support professional development. Region 1 ESC uses a 
combination of service center support plus other services offered through 
Connected Coaching and Connected University. Although the professional 
development models and providers differ, they all were expected to include 
some common required elements, such as support for immersion package 
components, the design of technology-enhanced learning environments 
and experiences, lesson development in the core-subject areas, sustained 
learning opportunities, and ongoing coaching and support. Individual dis- 
tricts and campuses collaborated with vendors to develop specific profes- 
sional development plans for their teachers and other staff. 

Technical and pedagogical support 

Each Technology Immersion package provider also is required to pro- 
vide campus-based technical support to advance the effective use of tech- 
nology for teaching and learning. Apple designed a Master Service and 
Support Program. Dell established a Call Center dedicated to technical 
support for TIP grantees as well as an 800 telephone number for hard- 
ware and software support. Region 1 ESC had an online and telephone 
HelpDesk to answer questions and provide assistance. 
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Appendix B 

Table B1 : Data Sources for Technology Immersion Implementation Indicators 


Indicator 

Source 

Item Description 

Index 

Score 

Standards- 
Based Score 

Leadership (all teachers) 

Cronbach's 
alpha = .97 
(spring 2007) 

Teacher 

survey 

07 7: Please indicate the extent of your agreement with each of 

the following statements. 

c) The principal consults with staff before making decisions 
about instructional technology that affect us. 

d) In this school there are clear expectations that technology 
will be used to enhance student learning. 

j) The principal in my school actively encourages teachers to 
pursue professional development geared towards curricular 
integration of technology. 

o) Our school has a well-developed technology plan that 
guides all technology integration efforts. 

p) The principal is an effective leader for instructional 
technology in this school. 

q) Overall, considering the uses of technology in my school 
today, 1 am confident that this use is leading to increased 
student achievement. 

r) The principal encourages teachers to be innovative and 
try new methods. 

t) The principal is willing to support — through funding or 
manpower — teachers' efforts at technology integration. 

v) Administrators in this school help teachers to use 
technology to access, analyze, and interpret student 
performance data 

w) Teachers receive adeguate administrative support to 
integrate technology into classroom practice. 

x) Teachers and administrators rely on research-proven 
teaching and learning principles in making decisions 
about technology use. 

y) When our school has professional development focused 
on technology, the principal often participates. 

5-point scale 
z score 

0 = Strongly Disagree 

1 = Disagree 

2 = Unsure 

3 = Agree 

4 = Strongly Agree 

Teacher Support (Innovative Culture) (all teachers) 

Cronbach's 
alpha = .82 

Teacher 

survey 

Qll: Please indicate the extent of your agreement with each of 
the following statements. 

b) Teachers in this school share an understanding about how 
technology will be used to enhance learning, 
i) Teachers in this school are continually learning and seeking 
new ideas. 

к) Teachers are not afraid to learn about new technologies and 
use them with their dass(es). 

аа) Teachers in this school are generally supportive of 
technology integration efforts. 

5-point scale 
z score 

0 = Strongly Disagree 

1 = Disagree 

2 = Unsure 

3 = Agree 

4 = Strongly Agree 

Parent & Community Support 

(all teachers) 

Cronbach's 
alpha = .84 

Teacher 

survey 

Qll: Please indicate the extent of your agreement with each of 
the following statements. 

f) Parents support our school's emphasis on technology, 
h) The surrounding community actively supports our 
instructional efforts with technology. 

5-point scale 
z score 

0 = Strongly Disagree 

1 = Disagree 

2 = Unsure 

3 = Agree 

4 = Strongly Agree 
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Table B1 : Data Sources for Technology Immersion Implementation Indicators 

(continued) 


Indicator 

Source 

Item Description 

Index 

Score 

Standards- 
Based Score 

Technical Support (all teachers) 

Cronbach's 
alpha = .67 

Teacher 

survey 

Qll: Please indicate the extent of your agreement with each of 

the following statements. 

a) Most of our school computers are kept in good working 
condition. 

b) Internet connections in my class are often too slow or not 
working. 

c) My reguests for technical assistance are addressed in a 
timely manner. 

d) Materials (e.g., software, printer supplies) for classroom 
use of computers are readily available in my school. 

e) Problems such as computers freezing or an inability 
to access the Internet make it difficult for me to use 
technology. 

5-point scale 
z score 

0 = Strongly Disagree 

1 = Disagree 

2 = Unsure 

3 = Agree 

4 = Strongly Agree 

Professional Development (core-subject teachers) 

Contact Hours 

Teacher 

survey 

Q20: Indicate the number of hours spent in technology-related 
professional development (PD) over the past school year 
(i.e., since August t, 2006). 

Continuous 
variable 0 
tox* 
z score 

Continuous variable 
Otox 

*>=35£)frommean 

excluded 

Classroom 

Support 

Cronbach's 
alpha = .67 

Teacher 

survey 

Ql 2: About how often do you interact with colleagues in each of 
the following ways. 

j) receive coaching or mentoring from an external 
(non-school) source such as a professional curriculum 
developer 

k) receive coaching or mentoring from an internal source, 
such as another teacher or technology coordinator 

5-point scale 
z score 

0 = Never 

1 = Rarely 

(a few times a year) 

2 = Sometimes (once 
or twice a month) 

3 = Often (once or 
twice a week) 

4 = Almost Daily 

Content Focus 

Cronbach's 
alpha = .92 

Teacher 

survey 

If core-subject teacher participated in technology-related PD, 
Q24: How much emphasis did the “most time"technology- 
related professional development activity give to each of the 
following areas? 

a) Curriculum (e.g., units, texts, standards) 

b) Instructional methods 

d) Lesson development in English language arts, mathematics, 
science, or social studies [mean of teachers' responses 
pertinent to their subject-area assignments 
(e.g., math teachers rate math)] 

3-point scale 
z score 

0 = No Emphasis 
2 = Minor Emphasis 
4 = Major Emphasis 
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Table B1 : Data Sources for Technology Immersion Implementation Indicators 

(continued) 


Indicator 

Source 

Item Description 

Index 

Score 

Standards- 
Based Score 

Professional Development (core-subject teachers) 

Coherence 

Cronbach's 
alpha = .92 

Teacher 

survey 

If core-subject teacher participated in technology-related PD, 

Q27: To what extent was the "most time" technology-related 
professional development activity: 

a) Consistent with your own goals for professional 
development 

b) Consistent with your school's or department's plan to 
change practice 

c) Based explicitly on what you had learned in earlier 
professional development experiences 

d) Followed up with activities that built upon what you 
learned in this professional development activity 

e) Designed to support state or district standards/curriculum 
frameworks 

f) Designed to support state or district assessment. 

5-point scale 
z score 

0 = Not at All 

1 
2 

3 

4 = Great Extent 

Classroom Immersion (core-subject teachers) 

Technology 

Integration 

Cronbach's 
alpha = .93 

Teacher 

survey 

Q 12: Please indicate your present level of classroom technology 

implementation. 

c) 1 alter my instructional use of the classroom computer(s) 
based upon the newest software applications and research 
on teaching, learning, and standards-based curriculum. 

d) My students discover innovative ways to use classroom 
computers to make a difference in their lives. 

e) 1 allocate time for students to practice their computer skills 
on the classroom computer(s). 

g) 1 integrate the most current research on teaching and 
learning when using the classroom computer(s). 

h) In my classroom, students use technology-based computer 
and Internet resources beyond the school (NASA, other 
government agencies, private sector) to solve authentic 
problems. 

i) My students' authentic problemsolving is supported by 
continuous access to a vast array of computer-based tools 
and technology. 

k) 1 plan computer-related activities in my classroom that will 
improve my students' basic skills (e.g., reading, writing, 
math computation). 

l) It is easy for me to design student-centered, integrated 
curriculum units that use the classroom computer(s) in a 
seamless fashion. 

n) 1 seek out activities that promote increased problem- 
solving and critical thinking using the classroom 
computer(s). 

o) Using cutting edge technology and computers, 1 have 
stretched the instructional computing in my classroom. 

7-point scale 
z score 

0 = Not true of me 
now 

1 = Somewhat true of 
me now 

2 = Somewhat true of 
me now 

3 = Somewhat true of 
me now 

4 = Very true of me 
now 
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Table B1 : Data Sources for Technology Immersion Implementation Indicators 

(continued) 


Indicator 

Source 

Item Description 

Index 

Score 

Standards- 
Based Score 

Classroom Immersion (core-subject teachers) 

Learner- 

Centered 

Instruction 

Cronbach's 
alpha = .81 

Teacher 

survey 

Q 12: Please indicate your present level of classroom technology 
implementation. 

b) Students authentic use of information and inquiry skills 
guides the type of instructional materials used in my 
classroom. 

j) My students are involved in establishing indivual goals 
within the classroom curriculum, 
m) In addition to traditional assessments, 1 consistently 
provide alternative assessment opportunities that 
encourage students to "showcase" their content 
understanding in nontraditional ways, 
q) My instructional approach emphasizes experiential 
learning, student involvement, and students solving 
"real-world" issues. 

7-point scale 
z score 

0 = Not true of me 
now 

1 = Somewhat true of 
me now 

2 = Somewhat true of 
me now 

3 = Somewhat true of 
me now 

4= Very true of me 
now 

Student 

Classroom 

Activities 

Cronbach's 
alpha = .90 

Teacher 

survey 

Q1 6: About how often do students in your typical class use 

technology in the following ways during class time. Students in 

my class use technology to... 

a) express themselves in writing (e.g., word processing). 

b) learn and practice skills (e.g., instructional software or 
educational games). 

c) enter, calculate, and graph information (e.g., Excel 
spreadsheet). 

d) create a database of information for a class project 
(e.g., Filemaker Pro, Access). 

e) create and make presentations (e.g., PowerPoint). 

f) communicate by email with peers, experts, or others on 
topics they are studying. 

h) conduct Internet research on an assigned topic. 

i) conduct multimedia research (reference CDs, online 
encyclopedias). 

j) enhance or express conceptual understanding through 
simulation/modeling software. 

k) visually represent or investigate concepts (e.g., through 
concept mapping, graphing, reading charts). 

l) produce print products (e.g., desktop publishing). 

m) produce multimedia reports/projects (e.g., with video, 
graphics, and sound editing). 

n) analyze information using tools such as graphing 
calculators or digital microscopes. 

p) complete a test or quiz (e.g., online assessments, Texas 
Math Diagnostic System). 

5-point scale 
z score 

0 = Never 
1.333 = Rarely 
(a few times a year) 
2.667 = Sometimes 
(once or twice a 
month) 

4 = Often (once 
or twice a week) 
or Almost Daily 
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Table B1 : Data Sources for Technology Immersion Implementation Indicators 

(continued) 


Indicator 

Source 

Item Description 

Index 

Score 

Standards- 
Based Score 

Classroom Immersion (core-subject teachers) 

Communication 

Cronbach's 
alpha = .74 

Teacher 

survey 

Q1 3: About how often do you use technology in each of the 
following ways? As a teacher 1. . . 

e) communicate with students. 

f) communicate with parents. 

g) communicate with colleagues/other professionals. 

m) post homework, class reguirements, or project information 
on a website. 

5-point scale 
z score 

0 = Never 

1 = Rarely (a few 
times a year) 

2 = Sometimes (once 
or twice a month) 

3 = Often (once or 
twice a week) 

4 = Almost Daily 

Professional 

Productivity 

Cronbach's 
alpha = . 85 

Teacher 

survey 

Q1 3: About how often do you use technology in each of the 
following ways? As a teacher 1. . . 

a) keep administrative records (e.g., attendance). 

b) manage student assessment data (e.g., electronic 
gradebooks). 

c) use technology to analyze and interpret student data 
to guide my instruction. 

d) create electronic lesson plans. 

h) create instructional materials (e.g., tests, handouts). 

i) gather information from the Internet to create a lesson 
(e.g., text, video, clipart). 

j) access model lesson plans integrating technology. 

k) deliver information using presentation software 
(e.g., PowerPoint). 

l) deliver information using multimedia presentations 
(text, audio, video, graphics). 

p) use the Internet at home for instructional purposes. 

g) use a computer to do schoolwork at home. 

5-point scale 
z score 

0 = Never 

1 = Rarely (a few 
times a year) 

2 = Sometimes (once 
or twice a month) 

3 = Often (once or 
twice a week) 

4 = Almost Daily 
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Table B1 : Data Sources for Technology Immersion Implementation Indicators 

(continued) 


Indicator 

Source 

Item Description 

Index 

Score 

Standards- 
Based Score 

Student Access and Use 

Laptop Access 
Days 

Student 

survey 

Q3.a: Does your school provide a laptop that you can use? 

[Yes = 180 days, No = 0 days] 

Q3.b: Have you had a laptop taken away from you for more than 
a class period? [No = 1 80 - 0 days; Yes = 180- Q3.d. no 
laptop days] 

Q3.d: How many days was the laptop taken away? [1 to 180] 

Continuous 
variable 
0 to 180 
z score 

Continuous variable 
0 to 180 

4 = Meet or exceed 
expectations 
0-3.99 = 

proportional fraction 
of requirement 
[campus mean 
adjusted for variance 
(-2 SDs)] 

Core Content 
Learning 

Cronbach's 
alpha = .73 

Student 

survey 

Q6: About how often do you use technology in each of the 
following classes? 

a) Reading/English language arts 

b) Math 

c) Science 

d) Social studies 

5-point scale 
z score 

0 = Never or Rarely (a 
few times a year) 
1.333 = Sometimes 
(once or twice a 
month) 

2.667 = Often (once 
or twice a week) 

4 = Almost Daily 

Home Learning 

Student 

survey 

Q4.a: How often can you take a laptop home? 

[0 = Never (no access); 1 = Only when 1 have a project or 
assignment or Other (restricted access) or As often as 1 want 
(full access)] 

Q4.b: When you take a laptop home, how do you use it? 
Homework for language arts (reading/writing) [+1] 
Homework for social studies [+1 ] 

Homework for science [+1] 

Homework for math [+1] 

Play games to learn [+1] 

Continuous 
variable 
0 to 6 
z score 

Continuous variable 
0 to 6 

0 = No access to 
laptop outside 
school 

1 = Restricted or 
full access to laptop 
outside school 

+ Laptop used for 
homework and/ 
or learning outside 
of school (up to 5 
points) 

4 = Meet or exceed 
expectations 
0-3.99 = 

proportional fraction 
of requirement 

Implementation Index 

Composite 
z score 
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Appendix C 

Table Cl : Scoring Rubrics for Measuring the Implementation Fidelity of 

Technology Immersion 


Component/ 

Element 

Minimal 

Immersion 

0-1.99 

Partial 

Immersion 

2.00-2.99 

Substantial 

Immersion 

3.00-3.49 

Full 

Immersion 

3.50-4.00 

Implementation 

Index 

Leadership 

Cam pus z Scores 


Teachers disagree 
or strongly disagree 
that administrators 
establish clear vision 
and expectations, 
encourage 
integration, provide 
supports, and involve 
staff in decisions. 

Teachers are unsure 
that administrators 
establish clear vision 
and expectations, 
encourage 
integration, provide 
supports, and involve 
staff indecisions. 

Teachers agree that 
administrators 
establish clear vision 
and expectations, 
encourage 
integration, provide 
supports, and involve 
staff in decisions. 

Teachers agree or 
strongly agree that 
administrators 
establish clear vision 
and expectations, 
encourage 
integration, provide 
supports, and involve 
staff indecisions. 


Teacher Support (Innovative Culture) 



Teachers disagree 
or strongly disagree 
that they share 
an understanding 
of technology, 
continually learn, are 
unafraid, and support 
integration. 

Teachers are unsure 
that they share 
an understanding 
of technology, 
continually learn, are 
unafraid, and support 
integration. 

Teachers agree 
that they share 
an understanding 
of technology, 
continually learn, are 
unafraid, and support 
integration. 

Teachers agree 
or strongly agree 
that they share 
an understanding 
of technology, 
continually learn, are 
unafraid, and support 
integration. 


Parent and Community Support 



Teachers disagree 
or strongly disagree 
that parents and 
the surrounding 
community support 
the school's efforts 
with technology. 

Teachers are unsure 
that parents and 
the surrounding 
community support 
the school's efforts 
with technology. 

Teachers agree 
that parents and 
the surrounding 
community support 
the school's efforts 
with technology. 

Teachers agree 
or strongly agree 
that parents and 
the surrounding 
community support 
the school's efforts 
with technology. 


Technical Support 



Teachers disagree 
or strongly disagree 
that computers are 
in good condition, 
Internet connections 
areadeguate, 
responses to 
reguests are timely, 
and materials are 
available. 

Teachers are unsure 
that computers are 
in good condition, 
Internet connections 
are adeguate, 
responses to 
reguests are timely, 
and materials are 
available. 

Teachers agree that 
computers are in 
good condition, 
Internet connections 
are adeguate, 
responses to 
reguests are timely, 
and materials are 
available. 

Teachers agree or 
strongly agree that 
computers are in 
good condition, 
Internet connections 
areadeguate, 
responses to 
reguests are timely, 
and materials are 
available. 
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Table Cl : Scoring Rubrics for Measuring the Implementation Fidelity of 

Technology Immersion (continued) 


Component/ 

Element 

Minimal 

Immersion 

0-1.99 

Partial 

Immersion 

2.00-2.99 

Substantial 

Immersion 

3.00-3.49 

Full 

Immersion 

3.50-4.00 

Implementation 

Index 

Professional Development 

Cam pus z Scores 

Contact Hours 

Core-subject 
teachers, on average, 
participated in 25 
or less hours of PD 
during the past 
school year. 

Core-subject 
teachers, on average, 
participated in 26 to 
37 hours of PD during 
the past school year. 

Core-subject 
teachers, on average, 
participated in 38 to 
49 hours of PD during 
the past school year. 

Core-subject 
teachers, on average, 
participated in 50 
or more hours of 
PD during the past 
school year. 


Classroom 

Support 

Core teachers indicate 
that they rarely 
or never receive 
classroom coaching 
or mentoring from an 
internal or external 
source. 

Core teachers indicate 
that they rarely (a few 
times a year) receive 
classroom coaching 
or mentoring from an 
internal or external 
source. 

Core teachers indicate 
that they sometimes 
(once or twice a 
month) receive 
classroom coaching 
or mentoring from an 
internal or external 
source. 

Core teachers indicate 
that they often (once 
or twice a week) or 
almost daily receive 
classroom coaching 
or mentoring from an 
internal or external 
source. 


Content Focus 

Core teachers indicate 
there is no or almost 
no PD emphasis 
on curriculum, 
instructional 
methods, and lesson 
development in core 
areas. 

Core teachers 
indicate there is a 
minor PD emphasis 
on curriculum, 
instructional 
methods, and lesson 
development in core 
areas. 

Core teachers indicate 
thereisam/norfo 
major PD emphasis 
on curriculum, 
instructional 
methods, and lesson 
development in core 
areas. 

Core teachers 
indicate there is a 
major PD emphasis 
on curriculum, 
instructional 
methods, and lesson 
development in core 
areas. 


Coherence 

Core teachers indicate 
that P D is not at 
all consistent with 
personal and school 
goals, prior learning, 
and state standards 
and assessment. 

Core teachers indicate 
that PD is consistent 
with personal and 
school goals, builds 
on prior learning, 
and supports state 
standards and 
assessment to a 
minimal extent. 

Core teachers indicate 
that PD is consistent 
with personal and 
school goals, builds 
on prior learning, 
and supports state 
standards and 
assessment to a 
moderate extent. 

Core teachers indicate 
that PD is consistent 
with personal and 
school goals, builds 
on prior learning, 
and supports state 
standards and 
assessment to a great 
extent. 
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Table Cl : Scoring Rubrics for Measuring the Implementation Fidelity of 

Technology Immersion (continued) 


Component/Ele- 

ment 

Minimal 

Immersion 

0-1.99 

Partial 

Immersion 

2.00-2.99 

Substantial 

Immersion 

3.00-3.49 

Full Immersion 
3.50-4.00 

Implementation 

Index 

Student Access and Use 

Campus z Scores 

Laptop Access 
Days 

Students' laptop 
access days vary to an 
extremely large extent 
at a campus, with 
laptops available 
from about 80 to 168 
days per student. 

Students' laptop 
access days vary to 
a large extent at a 
campus, with laptops 
available from about 
95 to 175 days per 
student. 

Students' laptop 
access days vary to a 
moderate extent at a 
campus, with laptops 
available from about 
140 to 175 days per 
student. 

Students' laptop 
access days vary to 
a small extent at a 
campus, with laptops 
available from about 
160 to 180 days per 
student. 


Core-Content 

Learning 

Students rarely (a 
few times a year) or 
never use technology 
resources in core- 
subject classes 

Students sometimes 
(once or twice a 
month) or often (once 
or twice a week) use 
technology resources 
in core-subject 
classes. 

Students often (once 
or twice a week) 
or almost daily use 
technology resources 
in core subjects. 

Students use 
technology resources 
in core subjects 
almost daily. 


Home Learning 

Students, on average, 
use their laptops 
outside of school 
for homework or 
learning either not 
at all or to a trivial 
extent. 

Students, on average 
use their laptops 
outside of school 
for homework and 
learning to a small 
extent. 

Students, on average, 
use their laptops 
outside of school 
for homework 
and learning to a 
moderate extent. 

Students, on average, 
use their laptops 
outside of school 
for homework and 
learning to a large 
extent. 
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Table Cl : Scoring Rubrics for Measuring the Implementation Fidelity of 

Technology Immersion (continued) 


Component/ 

Element 

Minimal 

Immersion 

0-1.99 

Partial 

Immersion 

2.00-2.99 

Substantial 

Immersion 

3.00-3.49 

Full 

Immersion 

3.50-4.00 

Implementation 

Index 

Classroom Immersion 

Campus z Scores 

Technology 

Integration 

Core teachers indicate 
it \s not true now that 
1 alter instructional 
practices, allocate 
time, integrate 
research on teaching 
and learning, improve 
basic skills, and 
support higher order 
thinking through 
technology. 

Core teachers indicate 
it is somewhat 
true now that 1 
alter instructional 
practices, allocate 
time, integrate 
research on teaching 
and learning, improve 
basic skills, and 
support higher order 
thinking through 
technology. 

Core teachers indicate 
it is somewhat or 
very true now that 
1 alter instructional 
practices, allocate 
time, integrate 
research on teaching 
and learning, improve 
basic skills, and 
support higher order 
thinking through 
technology. 

Core teachers indicate 
it is very true now that 
1 alter instructional 
practices, allocate 
time, integrate 
research on teaching 
and learning, improve 
basic skills, and 
support higher order 
thinking through 
technology. 


Learner- 

Centered 

Instruction 

Core teachers indicate 
it is not true now that 
my students establish 
learning goals, 
use information 
and inguiry skills, 
complete alternative 
assessments, and 
have active and 
relevant experiences. 

Core teachers indicate 
it is somewhat 
true now that my 
students establish 
learning goals, 
use information 
and inguiry skills, 
complete alternative 
assessments, and 
have active and 
relevant experiences. 

Core teachers indicate 
it is somewhat or very 
true now that my 
students establish 
learning goals, 
use information 
and inguiry skills, 
complete alternative 
assessments, and 
have active and 
relevant experiences. 

Core teachers indicate 
it is very true now that 
my students establish 
learning goals, 
use information 
and inguiry skills, 
complete alternative 
assessments, and 
have active and 
relevant experiences. 


Student 

Activities 

Core teachers rarely or 
never have students 
use technology 
resources to support 
core-content 
learning. 

Core teachers 
sometimes have 
students use 
technology resources 
to support core- 
content learning. 

Core teachers 
sometimes to often 
have students use 
technology resources 
to support core- 
content learning. 

Core teachers often 
to almost daily 
have students use 
technology resources 
to support core- 
content learning. 


Communication 

Core teachers rarely or 
never use technology 
to communicate with 
students, parents, 
and colleagues or to 
post information on a 
class website. 

Core teachers 
sometimes use 
technology to 
communicate with 
students, parents, 
and colleagues or to 
post information on a 
class website. 

Core teachers often 
use technology to 
communicate with 
students, parents, 
and colleagues or to 
post information on a 
class website. 

Core teachers often 
to almost daily 
use technology to 
communicate with 
students, parents, 
and colleagues or to 
post information on a 
class website. 


Professional 

Productivity 

Core teachers rarely or 
never use technology 
to enhance their 
professional 
productivity (e.g., 
keep records, analyze 
data, develop lessons, 
deliver information). 

Core teachers 
sometimes use 
technology to 
enhance their 
professional 
productivity (e.g., 
keep records, analyze 
data, develop lessons, 
deliver information). 

Core teachers often 
use technology 
to enhance their 
professional 
productivity (e.g., 
keep records, analyze 
data, develop lessons, 
deliver information). 

Core teachers often 
to almost daily 
use technology 
to enhance their 
professional 
productivity (e.g., 
keep records, analyze 
data, develop lessons, 
deliver information). 


Implementation Index 

Campus z Score 
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Endnotes 

1. Vernez, G., Karam, R., Mariano, L.T., & DeMartini, C. (2006). Evaluating 
Comprehensive School Reform Models at Scale: Focus on Implementation. 

Santa Monica, CA: RAND. 

2. Standards-based scores for Professional Development, Classroom Immersion, and 
Student Access and Use are averages across elements of these components. These 
scores serve descriptive purposes. Composite z scores were used in statistical 
analyses. 

3. Exploratory hierarchical regression analyses of the relationships between each 
of the elements of Immersion Support and each of the elements of Classroom 
Immersion, and students’ reading and mathematics achievement yielded 
inconsistent coefficients (positive and negative). Variables typically were not 
statistically significant predictors of test scores. 
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